Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-03-20T15:27:13Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-03-20T15:27:13Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260320 15:27:13.929986 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.62:43951
--webserver_interface=127.27.74.62
--webserver_port=0
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260320 15:27:14.288214 27952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:14.288583 27952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:14.288666 27952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260320 15:27:14.297801 27952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260320 15:27:14.297917 27952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:14.297973 27952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260320 15:27:14.298014 27952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260320 15:27:14.308657 27952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.62:43951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.74.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:14.310352 27952 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:14.312253 27952 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260320 15:27:14.322141 27957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:14.322125 27958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:14.322335 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:14.323206 27952 server_base.cc:1061] running on GCE node
I20260320 15:27:14.324378 27952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:14.327069 27952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:14.328359 27952 hybrid_clock.cc:648] HybridClock initialized: now 1774020434328277 us; error 74 us; skew 500 ppm
I20260320 15:27:14.328881 27952 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:14.331736 27952 webserver.cc:492] Webserver started at http://127.27.74.62:38201/ using document root <none> and password file <none>
I20260320 15:27:14.332645 27952 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:14.332772 27952 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:14.333261 27952 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:14.336483 27952 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/instance:
uuid: "ee1caebf640d4ec0bf9b28c4f496ca18"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.337513 27952 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal/instance:
uuid: "ee1caebf640d4ec0bf9b28c4f496ca18"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.344472 27952 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260320 15:27:14.349066 27966 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:14.350737 27952 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260320 15:27:14.350975 27952 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
uuid: "ee1caebf640d4ec0bf9b28c4f496ca18"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.351249 27952 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:14.377848 27952 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:14.378865 27952 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:14.379204 27952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:14.403966 27952 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.62:43951
I20260320 15:27:14.403990 28017 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.62:43951 every 8 connection(s)
I20260320 15:27:14.405706 27952 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
I20260320 15:27:14.410672 28018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:14.412497 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 27952
I20260320 15:27:14.412874 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal/instance
I20260320 15:27:14.415947 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.61:44739
--webserver_interface=127.27.74.61
--webserver_port=0
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260320 15:27:14.417610 28018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:14.442788 28018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
W20260320 15:27:14.444272 27967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
W20260320 15:27:14.446062 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.61:44739: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:14.495582 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } attempt: 1
W20260320 15:27:14.497714 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.61:44739: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:14.566100 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } attempt: 2
W20260320 15:27:14.568349 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.61:44739: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:14.659687 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } attempt: 3
W20260320 15:27:14.661903 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.61:44739: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
W20260320 15:27:14.783161 28020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:14.783640 28020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:14.783773 28020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260320 15:27:14.793474 28020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260320 15:27:14.793614 28020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:14.793751 28020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260320 15:27:14.793844 28020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260320 15:27:14.805296 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } attempt: 4
I20260320 15:27:14.805410 28020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.61:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.74.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
W20260320 15:27:14.807438 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.61:44739: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:14.807662 28020 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:14.809564 28020 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260320 15:27:14.818396 28028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:14.818769 28029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:14.819110 28031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:14.820003 28020 server_base.cc:1061] running on GCE node
I20260320 15:27:14.820927 28020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:14.822592 28020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:14.823833 28020 hybrid_clock.cc:648] HybridClock initialized: now 1774020434823786 us; error 55 us; skew 500 ppm
I20260320 15:27:14.824244 28020 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:14.827740 28020 webserver.cc:492] Webserver started at http://127.27.74.61:39103/ using document root <none> and password file <none>
I20260320 15:27:14.828655 28020 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:14.828811 28020 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:14.829246 28020 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:14.832199 28020 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/instance:
uuid: "07e8bb8819eb4cb4892e0cba1f758892"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.833204 28020 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal/instance:
uuid: "07e8bb8819eb4cb4892e0cba1f758892"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.840142 28020 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260320 15:27:14.844409 28037 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:14.845964 28020 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260320 15:27:14.846205 28020 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
uuid: "07e8bb8819eb4cb4892e0cba1f758892"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:14.846503 28020 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:14.886716 28020 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:14.887792 28020 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:14.888110 28020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:14.910970 28020 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.61:44739
I20260320 15:27:14.910928 28088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.61:44739 every 8 connection(s)
I20260320 15:27:14.912792 28020 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
I20260320 15:27:14.918066 28089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:14.919389 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28020
I20260320 15:27:14.919678 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal/instance
I20260320 15:27:14.921547 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.60:42563
--webserver_interface=127.27.74.60
--webserver_port=0
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260320 15:27:14.928467 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:14.944579 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:14.957978 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
W20260320 15:27:14.959328 28040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
W20260320 15:27:14.961333 28089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.010830 28089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 1
W20260320 15:27:15.013371 28089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.081836 28089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 2
W20260320 15:27:15.084254 28089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.106756 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } attempt: 5
I20260320 15:27:15.114023 28018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
W20260320 15:27:15.116073 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.167331 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 1
W20260320 15:27:15.170035 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.175613 28089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 3
W20260320 15:27:15.178846 28089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
I20260320 15:27:15.238318 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 2
W20260320 15:27:15.240497 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
W20260320 15:27:15.320539 28091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:15.320948 28091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:15.321075 28091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260320 15:27:15.322082 28089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 4
W20260320 15:27:15.324374 28089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
W20260320 15:27:15.330679 28091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260320 15:27:15.330834 28091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:15.330943 28091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260320 15:27:15.331025 28091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260320 15:27:15.341969 28091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.60:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.74.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:15.343904 28091 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:15.345960 28091 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260320 15:27:15.346740 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 3
W20260320 15:27:15.349110 28018 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.27.74.60:42563: Network error: Client connection negotiation failed: client connection to 127.27.74.60:42563: connect: Connection refused (error 111)
W20260320 15:27:15.355927 28099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:15.355973 28100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:15.357219 28102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:15.357766 28091 server_base.cc:1061] running on GCE node
I20260320 15:27:15.358676 28091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:15.360363 28091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:15.361646 28091 hybrid_clock.cc:648] HybridClock initialized: now 1774020435361598 us; error 49 us; skew 500 ppm
I20260320 15:27:15.362078 28091 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:15.364749 28091 webserver.cc:492] Webserver started at http://127.27.74.60:37629/ using document root <none> and password file <none>
I20260320 15:27:15.365577 28091 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:15.365761 28091 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:15.366174 28091 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:15.368896 28091 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data/instance:
uuid: "336af1a126ca4ab583c03527b890239a"
format_stamp: "Formatted at 2026-03-20 15:27:15 on dist-test-slave-qjxl"
I20260320 15:27:15.369764 28091 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal/instance:
uuid: "336af1a126ca4ab583c03527b890239a"
format_stamp: "Formatted at 2026-03-20 15:27:15 on dist-test-slave-qjxl"
I20260320 15:27:15.376223 28091 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260320 15:27:15.380587 28108 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:15.382077 28091 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260320 15:27:15.382287 28091 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal
uuid: "336af1a126ca4ab583c03527b890239a"
format_stamp: "Formatted at 2026-03-20 15:27:15 on dist-test-slave-qjxl"
I20260320 15:27:15.382531 28091 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:15.417681 28091 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:15.418684 28091 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:15.418982 28091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:15.441529 28091 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.60:42563
I20260320 15:27:15.441542 28159 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.60:42563 every 8 connection(s)
I20260320 15:27:15.443313 28091 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/data/info.pb
I20260320 15:27:15.444665 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28091
I20260320 15:27:15.444871 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-2/wal/instance
I20260320 15:27:15.448882 28160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:15.457765 28160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:15.472517 28160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:15.480947 28160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
I20260320 15:27:15.493831 28160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: Bootstrap starting.
I20260320 15:27:15.497785 28160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:15.498989 28160 log.cc:826] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:15.502781 28160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: No bootstrap required, opened a new log
I20260320 15:27:15.508082 28160 raft_consensus.cc:359] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.508419 28160 raft_consensus.cc:385] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:15.508513 28160 raft_consensus.cc:740] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 336af1a126ca4ab583c03527b890239a, State: Initialized, Role: FOLLOWER
I20260320 15:27:15.509323 28160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.510622 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.511005 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.512041 28160 sys_catalog.cc:565] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: configured and running, proceeding with master startup.
W20260320 15:27:15.523769 28181 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260320 15:27:15.523967 28181 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260320 15:27:15.520117 28091 master_runner.cc:186] Error getting master registration for 127.27.74.62:43951: OK, instance_id { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" instance_seqno: 1774020434398024 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260320 15:27:15.526433 28091 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260320 15:27:15.526469 28018 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 4
I20260320 15:27:15.538777 28018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Bootstrap starting.
I20260320 15:27:15.542382 28018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:15.543726 28018 log.cc:826] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:15.546722 28018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: No bootstrap required, opened a new log
I20260320 15:27:15.552954 28018 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.553292 28018 raft_consensus.cc:385] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:15.553435 28018 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Initialized, Role: FOLLOWER
I20260320 15:27:15.554396 28018 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.555505 28184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.555896 28184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.556793 28018 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: configured and running, proceeding with master startup.
W20260320 15:27:15.571177 28195 catalog_manager.cc:1568] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260320 15:27:15.571467 28195 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260320 15:27:15.569666 27952 master_runner.cc:186] Error getting master registration for 127.27.74.61:44739: OK, instance_id { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" instance_seqno: 1774020434904600 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260320 15:27:15.578275 27952 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260320 15:27:15.623674 28089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } attempt: 5
W20260320 15:27:15.632764 27972 tablet.cc:2379] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:15.637789 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Bootstrap starting.
I20260320 15:27:15.642376 28089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:15.644094 28089 log.cc:826] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:15.647809 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: No bootstrap required, opened a new log
I20260320 15:27:15.658265 28089 raft_consensus.cc:359] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.658710 28089 raft_consensus.cc:385] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:15.658843 28089 raft_consensus.cc:740] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e8bb8819eb4cb4892e0cba1f758892, State: Initialized, Role: FOLLOWER
I20260320 15:27:15.659797 28089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.661063 28198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.661525 28198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.662854 28089 sys_catalog.cc:565] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: configured and running, proceeding with master startup.
W20260320 15:27:15.674063 28114 tablet.cc:2379] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:15.684062 28184 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:15.684401 28184 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.686445 28184 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:15.688349 28135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:15.689018 28135 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 0.
I20260320 15:27:15.690105 28064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:15.690712 28064 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 0.
I20260320 15:27:15.691636 27969 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters:
I20260320 15:27:15.696772 28184 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:15.697016 28184 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:15.697115 28184 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:15.700940 28184 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:15.702486 28064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892"
I20260320 15:27:15.702749 28064 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:15.703222 28184 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 1 election: Requested vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:15.705014 28135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a"
I20260320 15:27:15.705262 28135 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:15.708461 28135 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 1.
I20260320 15:27:15.708209 28064 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 1.
I20260320 15:27:15.709129 27969 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters:
I20260320 15:27:15.709796 28184 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:15.719913 28020 master_runner.cc:263] No leader master found from master 07e8bb8819eb4cb4892e0cba1f758892
I20260320 15:27:15.720088 28020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260320 15:27:15.725356 28184 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 LEADER]: Becoming Leader. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Running, Role: LEADER
I20260320 15:27:15.726390 28184 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
W20260320 15:27:15.733259 28209 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260320 15:27:15.733453 28209 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260320 15:27:15.736696 28211 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.737124 28211 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:15.752254 28213 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260320 15:27:15.754964 28213 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260320 15:27:15.776528 28135 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Refusing update from remote peer ee1caebf640d4ec0bf9b28c4f496ca18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:15.777880 28211 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:15.788558 28064 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Refusing update from remote peer ee1caebf640d4ec0bf9b28c4f496ca18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:15.789933 28211 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:15.826489 28198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.826774 28198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.837450 28215 mvcc.cc:204] Tried to move back new op lower bound from 7266387704926023680 to 7266387704743059456. Current Snapshot: MvccSnapshot[applied={T|T < 7266387704926023680}]
I20260320 15:27:15.842188 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.842478 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.854887 28216 mvcc.cc:204] Tried to move back new op lower bound from 7266387704926023680 to 7266387704743059456. Current Snapshot: MvccSnapshot[applied={T|T < 7266387704926023680}]
I20260320 15:27:15.861111 28170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.861419 28170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.862459 28184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.862762 28184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:15.864003 28184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.864276 28184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:15.877424 28213 catalog_manager.cc:1357] Generated new cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:15.877580 28213 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260320 15:27:15.879513 28198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:15.879806 28198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:15.921224 28213 catalog_manager.cc:1380] Generated new certificate authority record
I20260320 15:27:15.924937 28213 catalog_manager.cc:1514] Loading token signing keys...
I20260320 15:27:15.947602 28213 catalog_manager.cc:6044] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Generated new TSK 0
I20260320 15:27:15.948982 28213 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260320 15:27:15.978286 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.74.1:0
--local_ip_for_outbound_sockets=127.27.74.1
--webserver_interface=127.27.74.1
--webserver_port=0
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20260320 15:27:16.448674 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:16.449088 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:16.449251 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260320 15:27:16.459923 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:16.460361 28222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.74.1
I20260320 15:27:16.471905 28222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.74.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.74.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.74.1
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:16.473816 28222 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:16.476037 28222 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260320 15:27:16.488896 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:16.488935 28227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:16.489832 28230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:16.490106 28222 server_base.cc:1061] running on GCE node
I20260320 15:27:16.491070 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:16.492409 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:16.493669 28222 hybrid_clock.cc:648] HybridClock initialized: now 1774020436493589 us; error 74 us; skew 500 ppm
I20260320 15:27:16.494117 28222 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:16.497495 28222 webserver.cc:492] Webserver started at http://127.27.74.1:36905/ using document root <none> and password file <none>
I20260320 15:27:16.498494 28222 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:16.498617 28222 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:16.499083 28222 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:16.502367 28222 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data/instance:
uuid: "3ce5383d28b347ab979765b880e5773c"
format_stamp: "Formatted at 2026-03-20 15:27:16 on dist-test-slave-qjxl"
I20260320 15:27:16.504038 28222 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/instance:
uuid: "3ce5383d28b347ab979765b880e5773c"
format_stamp: "Formatted at 2026-03-20 15:27:16 on dist-test-slave-qjxl"
I20260320 15:27:16.511696 28222 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260320 15:27:16.516913 28236 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:16.518656 28222 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260320 15:27:16.518893 28222 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal
uuid: "3ce5383d28b347ab979765b880e5773c"
format_stamp: "Formatted at 2026-03-20 15:27:16 on dist-test-slave-qjxl"
I20260320 15:27:16.519172 28222 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:16.526293 28181 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:16.526458 28181 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: loading cluster ID for follower catalog manager: success
I20260320 15:27:16.529721 28181 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: acquiring CA information for follower catalog manager: success
I20260320 15:27:16.532420 28181 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260320 15:27:16.549312 28222 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:16.550386 28222 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:16.550707 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:16.551838 28222 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260320 15:27:16.553853 28222 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260320 15:27:16.553992 28222 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:16.554124 28222 ts_tablet_manager.cc:616] Registered 0 tablets
I20260320 15:27:16.554206 28222 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:16.593722 28222 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.1:39627
I20260320 15:27:16.593789 28348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.1:39627 every 8 connection(s)
I20260320 15:27:16.595335 28222 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/data/info.pb
I20260320 15:27:16.596765 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28222
I20260320 15:27:16.597019 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/instance
I20260320 15:27:16.610399 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.74.2:0
--local_ip_for_outbound_sockets=127.27.74.2
--webserver_interface=127.27.74.2
--webserver_port=0
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260320 15:27:16.643290 28350 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:16.643308 28349 heartbeater.cc:344] Connected to a master server at 127.27.74.60:42563
I20260320 15:27:16.643605 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:16.643859 28351 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:16.643854 28350 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:16.643850 28349 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:16.645447 28350 heartbeater.cc:507] Master 127.27.74.62:43951 requested a full tablet report, sending...
I20260320 15:27:16.645500 28351 heartbeater.cc:507] Master 127.27.74.61:44739 requested a full tablet report, sending...
I20260320 15:27:16.645537 28349 heartbeater.cc:507] Master 127.27.74.60:42563 requested a full tablet report, sending...
I20260320 15:27:16.648234 27983 ts_manager.cc:194] Registered new tserver with Master: 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:16.648234 28054 ts_manager.cc:194] Registered new tserver with Master: 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:16.648372 28125 ts_manager.cc:194] Registered new tserver with Master: 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:16.650167 27983 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.74.1:47055
I20260320 15:27:16.735509 28209 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:16.735707 28209 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: loading cluster ID for follower catalog manager: success
I20260320 15:27:16.738500 28209 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: acquiring CA information for follower catalog manager: success
I20260320 15:27:16.740562 28209 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260320 15:27:16.898393 28360 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:16.898666 28360 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:16.900206 28360 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
I20260320 15:27:16.900871 28063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:16.901065 27993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:16.902110 28112 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:16.902774 28360 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:16.908572 28361 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:16.908829 28361 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:16.910269 28361 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:16.910856 27993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:16.911018 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:16.911281 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07e8bb8819eb4cb4892e0cba1f758892 in term 1.
I20260320 15:27:16.911933 28040 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892, 336af1a126ca4ab583c03527b890239a; no voters: ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:16.912467 28361 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:16.912650 28361 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:16.912837 28361 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:16.916021 28361 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:16.916800 28361 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 2 election: Requested vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:16.917514 27993 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18"
I20260320 15:27:16.917944 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a"
I20260320 15:27:16.918155 28134 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:16.921276 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07e8bb8819eb4cb4892e0cba1f758892 in term 2.
I20260320 15:27:16.921901 28040 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892, 336af1a126ca4ab583c03527b890239a; no voters: ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:16.922304 28361 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:16.922833 28361 raft_consensus.cc:697] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 LEADER]: Becoming Leader. State: Replica: 07e8bb8819eb4cb4892e0cba1f758892, State: Running, Role: LEADER
I20260320 15:27:16.923466 28361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:16.926203 28362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e8bb8819eb4cb4892e0cba1f758892. Latest consensus state: current_term: 2 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:16.926527 28362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:16.928243 28363 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260320 15:27:16.931185 28363 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260320 15:27:16.932549 28363 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:16.932687 28363 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260320 15:27:16.934463 28363 catalog_manager.cc:1514] Loading token signing keys...
I20260320 15:27:16.936190 28363 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Loaded TSK: 0
I20260320 15:27:16.938099 28363 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260320 15:27:16.977972 28063 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 LEADER]: Rejecting Update request from peer ee1caebf640d4ec0bf9b28c4f496ca18 for earlier term 1. Current term is 2. Ops: []
I20260320 15:27:16.979122 28364 consensus_queue.cc:1059] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260320 15:27:16.979895 28364 consensus_queue.cc:1243] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Peer 07e8bb8819eb4cb4892e0cba1f758892 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260320 15:27:16.980057 28365 raft_consensus.cc:3055] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 LEADER]: Stepping down as leader of term 1
I20260320 15:27:16.980151 28365 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Running, Role: LEADER
I20260320 15:27:16.980538 28365 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:16.981084 28365 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 1 FOLLOWER]: Advancing to term 2
W20260320 15:27:17.013710 28359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:17.014461 28359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:17.014644 28359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260320 15:27:17.024202 28359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:17.024432 28359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.74.2
I20260320 15:27:17.035702 28359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.74.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.74.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.74.2
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:17.037565 28359 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:17.039630 28359 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260320 15:27:17.051174 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:17.051137 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:17.051546 28370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:17.052987 28359 server_base.cc:1061] running on GCE node
I20260320 15:27:17.053660 28359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:17.054831 28359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:17.056105 28359 hybrid_clock.cc:648] HybridClock initialized: now 1774020437056045 us; error 45 us; skew 500 ppm
I20260320 15:27:17.056520 28359 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:17.058965 28359 webserver.cc:492] Webserver started at http://127.27.74.2:37759/ using document root <none> and password file <none>
I20260320 15:27:17.059934 28359 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:17.060081 28359 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:17.060503 28359 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:17.063274 28359 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data/instance:
uuid: "d22e73471b9046d28d3cf63c905b11b7"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.064430 28359 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/instance:
uuid: "d22e73471b9046d28d3cf63c905b11b7"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.070912 28359 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260320 15:27:17.076573 28379 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.078486 28359 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260320 15:27:17.078688 28359 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal
uuid: "d22e73471b9046d28d3cf63c905b11b7"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.079008 28359 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:17.102247 28359 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:17.103302 28359 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:17.103667 28359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:17.104755 28359 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260320 15:27:17.106678 28359 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260320 15:27:17.106777 28359 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.106910 28359 ts_tablet_manager.cc:616] Registered 0 tablets
I20260320 15:27:17.107134 28359 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.148310 28359 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.2:45233
I20260320 15:27:17.148408 28491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.2:45233 every 8 connection(s)
I20260320 15:27:17.150059 28359 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/data/info.pb
I20260320 15:27:17.153249 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:17.159561 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28359
I20260320 15:27:17.159808 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/instance
I20260320 15:27:17.169857 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.74.3:0
--local_ip_for_outbound_sockets=127.27.74.3
--webserver_interface=127.27.74.3
--webserver_port=0
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260320 15:27:17.198195 28492 heartbeater.cc:344] Connected to a master server at 127.27.74.60:42563
I20260320 15:27:17.198693 28492 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.199012 28493 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:17.199172 28493 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.200013 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:17.200178 28494 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.200167 28492 heartbeater.cc:507] Master 127.27.74.60:42563 requested a full tablet report, sending...
I20260320 15:27:17.200259 28493 heartbeater.cc:507] Master 127.27.74.62:43951 requested a full tablet report, sending...
I20260320 15:27:17.200896 28494 heartbeater.cc:507] Master 127.27.74.61:44739 requested a full tablet report, sending...
I20260320 15:27:17.202371 28125 ts_manager.cc:194] Registered new tserver with Master: d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:17.202361 28054 ts_manager.cc:194] Registered new tserver with Master: d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:17.202433 27983 ts_manager.cc:194] Registered new tserver with Master: d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:17.204514 28054 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.74.2:58261
I20260320 15:27:17.455897 28365 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:17.456149 28365 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:17.457116 28365 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:17.457736 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:17.457744 28063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:17.458045 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 2.
I20260320 15:27:17.458536 27969 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters: 07e8bb8819eb4cb4892e0cba1f758892
I20260320 15:27:17.458894 28365 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260320 15:27:17.459018 28365 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:17.459136 28365 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 2 FOLLOWER]: Advancing to term 3
I20260320 15:27:17.461516 28365 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:17.462319 28365 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 3 election: Requested vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:17.462975 28063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892"
I20260320 15:27:17.463023 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a"
I20260320 15:27:17.463289 28134 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 2 FOLLOWER]: Advancing to term 3
I20260320 15:27:17.466599 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 3.
I20260320 15:27:17.467319 27969 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters: 07e8bb8819eb4cb4892e0cba1f758892
I20260320 15:27:17.467831 28365 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 FOLLOWER]: Leader election won for term 3
I20260320 15:27:17.468045 28365 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 LEADER]: Becoming Leader. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Running, Role: LEADER
I20260320 15:27:17.468469 28365 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:17.470106 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.470405 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:17.470980 28504 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260320 15:27:17.473292 28134 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Rejecting Update request from peer 07e8bb8819eb4cb4892e0cba1f758892 for earlier term 2. Current term is 3. Ops: []
I20260320 15:27:17.473759 28504 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260320 15:27:17.474285 28505 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260320 15:27:17.474858 28504 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:17.474969 28504 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260320 15:27:17.476043 28505 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 LEADER]: Stepping down as leader of term 2
I20260320 15:27:17.476205 28505 raft_consensus.cc:740] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07e8bb8819eb4cb4892e0cba1f758892, State: Running, Role: LEADER
I20260320 15:27:17.476802 28504 catalog_manager.cc:1514] Loading token signing keys...
I20260320 15:27:17.476678 28505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:17.477316 28505 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 2 FOLLOWER]: Advancing to term 3
I20260320 15:27:17.477923 28504 catalog_manager.cc:6055] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Loaded TSK: 0
I20260320 15:27:17.478731 28504 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20260320 15:27:17.558266 28502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:17.558602 28502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:17.558689 28502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260320 15:27:17.567860 28502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:17.568094 28502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.74.3
I20260320 15:27:17.578680 28502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.74.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.74.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.74.3
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:17.580437 28502 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:17.582296 28502 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260320 15:27:17.593039 28511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:17.593081 28512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:17.594082 28514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:17.594404 28502 server_base.cc:1061] running on GCE node
I20260320 15:27:17.595079 28502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:17.596208 28502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:17.597488 28502 hybrid_clock.cc:648] HybridClock initialized: now 1774020437597434 us; error 61 us; skew 500 ppm
I20260320 15:27:17.597878 28502 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:17.600934 28502 webserver.cc:492] Webserver started at http://127.27.74.3:46417/ using document root <none> and password file <none>
I20260320 15:27:17.601743 28502 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:17.601840 28502 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:17.602164 28502 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260320 15:27:17.604882 28502 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data/instance:
uuid: "80636bb608a1431b955576e296ff63bb"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.605717 28502 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/instance:
uuid: "80636bb608a1431b955576e296ff63bb"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.611671 28502 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260320 15:27:17.615622 28520 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.617326 28502 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:17.617568 28502 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal
uuid: "80636bb608a1431b955576e296ff63bb"
format_stamp: "Formatted at 2026-03-20 15:27:17 on dist-test-slave-qjxl"
I20260320 15:27:17.617808 28502 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:17.639858 28502 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:17.641175 28502 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:17.641520 28502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:17.642642 28502 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260320 15:27:17.644665 28502 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260320 15:27:17.644765 28502 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.644896 28502 ts_tablet_manager.cc:616] Registered 0 tablets
I20260320 15:27:17.644971 28502 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:17.653755 28351 heartbeater.cc:499] Master 127.27.74.61:44739 was elected leader, sending a full tablet report...
I20260320 15:27:17.683159 28502 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.3:44463
I20260320 15:27:17.683287 28632 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.3:44463 every 8 connection(s)
I20260320 15:27:17.684814 28502 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/data/info.pb
I20260320 15:27:17.686939 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28502
I20260320 15:27:17.687188 27944 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/instance
I20260320 15:27:17.707481 28494 heartbeater.cc:499] Master 127.27.74.61:44739 was elected leader, sending a full tablet report...
I20260320 15:27:17.716981 28634 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:17.717058 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:17.717470 28635 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.717478 28634 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.717550 28633 heartbeater.cc:344] Connected to a master server at 127.27.74.60:42563
I20260320 15:27:17.717718 28633 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:17.719089 28633 heartbeater.cc:507] Master 127.27.74.60:42563 requested a full tablet report, sending...
I20260320 15:27:17.719154 28635 heartbeater.cc:507] Master 127.27.74.61:44739 requested a full tablet report, sending...
I20260320 15:27:17.720165 28634 heartbeater.cc:507] Master 127.27.74.62:43951 requested a full tablet report, sending...
I20260320 15:27:17.721150 28124 ts_manager.cc:194] Registered new tserver with Master: 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:17.721246 28054 ts_manager.cc:194] Registered new tserver with Master: 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:17.721354 27983 ts_manager.cc:194] Registered new tserver with Master: 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:17.722851 27983 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.27.74.3:44729
I20260320 15:27:17.740023 27944 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260320 15:27:17.777784 27983 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_f77f3a040dcf4652b889e6c431aa66e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.779445 27983 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f77f3a040dcf4652b889e6c431aa66e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.782531 27982 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_95267f495f214d8b9fd4654117ca0cd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.783265 27982 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95267f495f214d8b9fd4654117ca0cd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.784827 28134 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Refusing update from remote peer ee1caebf640d4ec0bf9b28c4f496ca18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260320 15:27:17.785483 28063 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Refusing update from remote peer ee1caebf640d4ec0bf9b28c4f496ca18: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260320 15:27:17.785701 27981 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_051279c04a194666aa02412a3e5f7b69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.786515 27981 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_051279c04a194666aa02412a3e5f7b69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.786697 28364 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260320 15:27:17.786969 28364 consensus_queue.cc:1243] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Peer 07e8bb8819eb4cb4892e0cba1f758892 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260320 15:27:17.788184 28364 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260320 15:27:17.789377 27980 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_1187b07e05cc4a1bb3df0e2939ca38c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.790256 27980 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1187b07e05cc4a1bb3df0e2939ca38c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.790755 28063 pending_rounds.cc:85] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Aborting all ops after (but not including) 4
I20260320 15:27:17.790597 27977 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_27381ad91a994e2fa959d8cc5b035e25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:17.790970 28063 pending_rounds.cc:107] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260320 15:27:17.791079 28063 raft_consensus.cc:2889] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260320 15:27:17.791574 27977 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27381ad91a994e2fa959d8cc5b035e25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.792001 27978 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_b277cf811d544296a4f6507a9a0a3423"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.792866 27978 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b277cf811d544296a4f6507a9a0a3423 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.793682 27975 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_4ccd35d52c304c08bab71d80c7f76469"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.794682 27975 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ccd35d52c304c08bab71d80c7f76469 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.794828 27976 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_2b57218449c645c8a2fad890a1551010"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.795732 27976 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b57218449c645c8a2fad890a1551010 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.790271 27979 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_a8310d1726fa4279ab8f257a90183227"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.799065 27979 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8310d1726fa4279ab8f257a90183227 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.800305 28505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.800573 28505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:17.800659 27974 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_6d703cd3356d419c97e2127c89d5f534"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:17.801350 27974 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d703cd3356d419c97e2127c89d5f534 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:17.815301 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.815582 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:17.815040 28658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.816295 28658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:17.816344 27983 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_95267f495f214d8b9fd4654117ca0cd5" } new_table_name: "default.Table_20f1ffa482184f37bae21b551b847cd2" modify_external_catalogs: true
I20260320 15:27:17.817849 28505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.818087 28505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:17.818755 27982 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_f77f3a040dcf4652b889e6c431aa66e6" } new_table_name: "default.Table_62e5b7142f9b495e9980315428909e81" modify_external_catalogs: true
I20260320 15:27:17.824867 27981 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_051279c04a194666aa02412a3e5f7b69" } new_table_name: "default.Table_84d3feb7c70543a389a534f94bd0c6ba" modify_external_catalogs: true
I20260320 15:27:17.830823 28657 mvcc.cc:204] Tried to move back new op lower bound from 7266387713243484160 to 7266387711873486848. Current Snapshot: MvccSnapshot[applied={T|T < 7266387713243484160 or (T in {7266387713243484160})}]
I20260320 15:27:17.863224 27975 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_27381ad91a994e2fa959d8cc5b035e25" } new_table_name: "default.Table_cc6df4082b8248a7bd78d01689bd8a57" modify_external_catalogs: true
I20260320 15:27:17.864921 27978 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_4ccd35d52c304c08bab71d80c7f76469" } modify_external_catalogs: true
I20260320 15:27:17.865123 27978 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_4ccd35d52c304c08bab71d80c7f76469" } modify_external_catalogs: true
I20260320 15:27:17.866854 27980 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_1187b07e05cc4a1bb3df0e2939ca38c3" } new_table_name: "default.Table_cc65451d98cc4f7187cec6ec36e20a7d" modify_external_catalogs: true
I20260320 15:27:17.868340 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.868602 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:17.872387 28658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:17.872696 28658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:17.873323 27977 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_b277cf811d544296a4f6507a9a0a3423" } modify_external_catalogs: true
I20260320 15:27:17.873572 27977 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_b277cf811d544296a4f6507a9a0a3423" } modify_external_catalogs: true
I20260320 15:27:17.878317 27974 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_6d703cd3356d419c97e2127c89d5f534" } modify_external_catalogs: true
I20260320 15:27:17.879897 27974 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_6d703cd3356d419c97e2127c89d5f534" } modify_external_catalogs: true
I20260320 15:27:17.878654 27976 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_a8310d1726fa4279ab8f257a90183227" } modify_external_catalogs: true
I20260320 15:27:17.880599 27976 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_a8310d1726fa4279ab8f257a90183227" } modify_external_catalogs: true
I20260320 15:27:17.881743 27979 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_2b57218449c645c8a2fad890a1551010" } modify_external_catalogs: true
I20260320 15:27:17.882289 27979 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "default.Table_2b57218449c645c8a2fad890a1551010" } modify_external_catalogs: true
W20260320 15:27:17.883358 27983 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4346c464fb34e9dae8aeccc99dc07a3 has no consensus state
W20260320 15:27:17.884517 27982 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08181eea58a24e1cb424f0e1e10f46ca has no consensus state
W20260320 15:27:17.890260 27981 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db has no consensus state
I20260320 15:27:17.890265 27944 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskc3SESu/build/asan/bin/kudu pid 27952
W20260320 15:27:17.890460 27975 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 88a005b1c2b54a2fa8515f7a6b165ea8 has no consensus state
I20260320 15:27:17.891872 28013 generic_service.cc:196] Checking for leaks (request via RPC)
W20260320 15:27:17.899437 27980 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6ce1fe5280a4e888b8f1bec19ab432f has no consensus state
I20260320 15:27:18.022014 27974 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 3947f9a144644fc5aaf4dd524bac3a02
I20260320 15:27:18.022511 27976 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 36ddb28888ac444180ee355c293a0289
W20260320 15:27:18.021562 27967 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet b4346c464fb34e9dae8aeccc99dc07a3 (table default.Table_20f1ffa482184f37bae21b551b847cd2 [id=2b3e6ef0dd14451cbcb012d550246753], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b4346c464fb34e9dae8aeccc99dc07a3 consensus state has no leader
I20260320 15:27:18.022722 27977 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 98128a8fa8cf4fd9b1108946a4807b3b
I20260320 15:27:18.022948 27978 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 94a8b5073e134b248dadf47f10353951
I20260320 15:27:18.027477 27976 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_96f581b9cc2f462faec112e20e287179"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.027878 27977 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_ebc7ffacd75e49a4b2ed94aafff57936"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.028422 27976 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96f581b9cc2f462faec112e20e287179 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.028560 27977 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebc7ffacd75e49a4b2ed94aafff57936 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.029608 27978 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.030242 27978 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.031806 27974 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "default.Table_54061406504947f48bb5bf99514ead4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.032581 27974 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54061406504947f48bb5bf99514ead4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.034547 27944 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskc3SESu/build/asan/bin/kudu with pid 27952
W20260320 15:27:18.057924 28111 connection.cc:537] server connection from 127.0.0.1:49404 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:18.059238 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0320 15:27:17.893016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.27.74.1:39627)
0320 15:27:17.894189 (+ 1173us) server_negotiation.cc:207] Beginning negotiation
0320 15:27:17.894204 (+ 15us) server_negotiation.cc:400] Waiting for connection header
0320 15:27:18.034190 (+139986us) server_negotiation.cc:408] Connection header received
0320 15:27:18.034711 (+ 521us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0320 15:27:18.034723 (+ 12us) server_negotiation.cc:462] Received NEGOTIATE request from client
0320 15:27:18.035066 (+ 343us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0320 15:27:18.035287 (+ 221us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
0320 15:27:18.057956 (+ 22669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57818: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":901,"threads_started":1}
W20260320 15:27:18.059397 28680 negotiation.cc:337] Failed RPC negotiation. Trace:
0320 15:27:17.892949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.27.74.3:44463)
0320 15:27:17.894261 (+ 1312us) server_negotiation.cc:207] Beginning negotiation
0320 15:27:17.894270 (+ 9us) server_negotiation.cc:400] Waiting for connection header
0320 15:27:18.057944 (+163674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51056: BlockingRecv error: recv got EOF from 127.0.0.1:51056 (error 108)
Metrics: {"server-negotiator.queue_time_us":1130}
W20260320 15:27:18.059476 28701 negotiation.cc:337] Failed RPC negotiation. Trace:
0320 15:27:18.026690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.27.74.2:45233)
0320 15:27:18.033354 (+ 6664us) server_negotiation.cc:207] Beginning negotiation
0320 15:27:18.033364 (+ 10us) server_negotiation.cc:400] Waiting for connection header
0320 15:27:18.058272 (+ 24908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39576: BlockingRecv error: recv got EOF from 127.0.0.1:39576 (error 108)
Metrics: {"server-negotiator.queue_time_us":6487,"thread_start_us":6381,"threads_started":1}
W20260320 15:27:18.060343 28643 connection.cc:537] client connection to 127.27.74.62:43951 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:18.061359 28643 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.74.62:43951)
W20260320 15:27:18.062270 28643 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.27.74.62:43951)
W20260320 15:27:18.086104 28643 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.27.74.62:43951)
I20260320 15:27:18.157291 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:18.158823 28350 heartbeater.cc:646] Failed to heartbeat to 127.27.74.62:43951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.207012 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:18.208871 28493 heartbeater.cc:646] Failed to heartbeat to 127.27.74.62:43951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.225759 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:18.227674 28634 heartbeater.cc:646] Failed to heartbeat to 127.27.74.62:43951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.396641 28705 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:18.396886 28705 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:18.397769 28705 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
I20260320 15:27:18.398428 28063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 4 candidate_status { last_received { term: 3 index: 25 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
W20260320 15:27:18.399044 28112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
W20260320 15:27:18.401002 28112 leader_election.cc:336] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951): Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.401162 28112 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:18.401530 28705 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260320 15:27:18.526110 28708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:18.526418 28708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:18.527748 28708 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:18.527810 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 4 candidate_status { last_received { term: 3 index: 26 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:18.528064 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07e8bb8819eb4cb4892e0cba1f758892 in term 3.
I20260320 15:27:18.528587 28040 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892, 336af1a126ca4ab583c03527b890239a; no voters:
I20260320 15:27:18.528891 28708 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260320 15:27:18.529018 28708 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:18.529129 28708 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 3 FOLLOWER]: Advancing to term 4
W20260320 15:27:18.529780 28041 leader_election.cc:336] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951): Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.532330 28708 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:18.532892 28708 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 election: Requested vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:18.533622 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 4 candidate_status { last_received { term: 3 index: 26 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a"
I20260320 15:27:18.533830 28134 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 3 FOLLOWER]: Advancing to term 4
W20260320 15:27:18.534579 28041 leader_election.cc:336] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951): Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111)
I20260320 15:27:18.535861 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07e8bb8819eb4cb4892e0cba1f758892 in term 4.
I20260320 15:27:18.536393 28040 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892, 336af1a126ca4ab583c03527b890239a; no voters: ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:18.536752 28708 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Leader election won for term 4
I20260320 15:27:18.537146 28708 raft_consensus.cc:697] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 LEADER]: Becoming Leader. State: Replica: 07e8bb8819eb4cb4892e0cba1f758892, State: Running, Role: LEADER
I20260320 15:27:18.537504 28708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:18.538358 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e8bb8819eb4cb4892e0cba1f758892. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:18.538712 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:18.539366 28712 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260320 15:27:18.541814 28712 catalog_manager.cc:679] Loaded metadata for table default.Table_20f1ffa482184f37bae21b551b847cd2 [id=2b3e6ef0dd14451cbcb012d550246753]
I20260320 15:27:18.542351 28712 catalog_manager.cc:679] Loaded metadata for table default.Table_62e5b7142f9b495e9980315428909e81 [id=38d526902b62496087e2bbcb5856b395]
I20260320 15:27:18.542738 28712 catalog_manager.cc:679] Loaded metadata for table default.Table_cc65451d98cc4f7187cec6ec36e20a7d [id=4494ebcc34f1468c9ce7381b5e5ab7d6]
I20260320 15:27:18.543406 28712 catalog_manager.cc:679] Loaded metadata for table default.Table_cc6df4082b8248a7bd78d01689bd8a57 [id=634e24fdc81640fbb393a4b19bac7343]
I20260320 15:27:18.544270 28712 catalog_manager.cc:679] Loaded metadata for table default.Table_84d3feb7c70543a389a534f94bd0c6ba [id=d3714568b2244ef1ab538461b3a730f8]
I20260320 15:27:18.546123 28712 tablet_loader.cc:96] loaded metadata for tablet 08181eea58a24e1cb424f0e1e10f46ca (table default.Table_62e5b7142f9b495e9980315428909e81 [id=38d526902b62496087e2bbcb5856b395])
I20260320 15:27:18.546499 28712 tablet_loader.cc:96] loaded metadata for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_84d3feb7c70543a389a534f94bd0c6ba [id=d3714568b2244ef1ab538461b3a730f8])
I20260320 15:27:18.547030 28712 tablet_loader.cc:96] loaded metadata for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (table default.Table_cc6df4082b8248a7bd78d01689bd8a57 [id=634e24fdc81640fbb393a4b19bac7343])
I20260320 15:27:18.547808 28712 tablet_loader.cc:96] loaded metadata for tablet b4346c464fb34e9dae8aeccc99dc07a3 (table default.Table_20f1ffa482184f37bae21b551b847cd2 [id=2b3e6ef0dd14451cbcb012d550246753])
I20260320 15:27:18.548350 28712 tablet_loader.cc:96] loaded metadata for tablet d6ce1fe5280a4e888b8f1bec19ab432f (table default.Table_cc65451d98cc4f7187cec6ec36e20a7d [id=4494ebcc34f1468c9ce7381b5e5ab7d6])
I20260320 15:27:18.548655 28712 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260320 15:27:18.550278 28712 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:18.550357 28712 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260320 15:27:18.552017 28712 catalog_manager.cc:1514] Loading token signing keys...
I20260320 15:27:18.553020 28712 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Loaded TSK: 0
I20260320 15:27:18.553468 28712 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260320 15:27:18.585726 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_752d88f1a66e44b49d0d3e91e2c5786c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.587092 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4b0fd934c3e94b18b763f7587051eacf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.587878 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.587634 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_96f581b9cc2f462faec112e20e287179"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.586402 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2fdea31f3e72453d949a79a1daf3d9b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.585987 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_54061406504947f48bb5bf99514ead4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.586045 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_11458e6c1def469191e80286da6a22c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.587381 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_8567ed8118d64bda8110f902052fc1f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.586735 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ebc7ffacd75e49a4b2ed94aafff57936"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.586853 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2b57218449c645c8a2fad890a1551010" } modify_external_catalogs: true
W20260320 15:27:18.593101 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11458e6c1def469191e80286da6a22c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593376 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b0fd934c3e94b18b763f7587051eacf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593709 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_96f581b9cc2f462faec112e20e287179 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593806 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593902 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54061406504947f48bb5bf99514ead4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593101 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8567ed8118d64bda8110f902052fc1f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.593101 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebc7ffacd75e49a4b2ed94aafff57936 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.594316 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fdea31f3e72453d949a79a1daf3d9b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.594367 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_752d88f1a66e44b49d0d3e91e2c5786c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.594513 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2b57218449c645c8a2fad890a1551010" } modify_external_catalogs: true
I20260320 15:27:18.595837 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f10f448e49864a74923ce64809178a36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.596601 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f10f448e49864a74923ce64809178a36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.602151 28134 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Refusing update from remote peer 07e8bb8819eb4cb4892e0cba1f758892: Log matching property violated. Preceding OpId in replica: term: 3 index: 25. Preceding OpId from leader: term: 4 index: 28. (index mismatch)
I20260320 15:27:18.603147 28714 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 20, Time since last communication: 0.000s
W20260320 15:27:18.603591 28041 consensus_peers.cc:597] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 -> Peer ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951): Couldn't send request to peer ee1caebf640d4ec0bf9b28c4f496ca18. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:18.609341 28705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e8bb8819eb4cb4892e0cba1f758892. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:18.609654 28705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:18.612164 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:18.612444 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:18.614012 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cc65451d98cc4f7187cec6ec36e20a7d" } new_table_name: "default.Table_066f88ea62914d6f93053e540c6eae61" modify_external_catalogs: true
I20260320 15:27:18.616801 28705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:18.617051 28705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:18.623931 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f10f448e49864a74923ce64809178a36" } modify_external_catalogs: true
I20260320 15:27:18.624240 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f10f448e49864a74923ce64809178a36" } modify_external_catalogs: true
I20260320 15:27:18.625360 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_84d3feb7c70543a389a534f94bd0c6ba" } new_table_name: "default.Table_549f77fe41f947ab8e14fbce182049f8" modify_external_catalogs: true
I20260320 15:27:18.625458 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cc6df4082b8248a7bd78d01689bd8a57" } new_table_name: "default.Table_947a59a4bb104546b3b24ef82815ccec" modify_external_catalogs: true
I20260320 15:27:18.626060 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_20f1ffa482184f37bae21b551b847cd2" } new_table_name: "default.Table_2668da52d69e491eab99d1abc6909b41" modify_external_catalogs: true
I20260320 15:27:18.626188 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_54061406504947f48bb5bf99514ead4a" } modify_external_catalogs: true
I20260320 15:27:18.626385 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_54061406504947f48bb5bf99514ead4a" } modify_external_catalogs: true
I20260320 15:27:18.626398 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_62e5b7142f9b495e9980315428909e81" } new_table_name: "default.Table_05ab9b1188db49fdb97c2cd0f3acadbe" modify_external_catalogs: true
I20260320 15:27:18.627115 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71" } modify_external_catalogs: true
I20260320 15:27:18.627316 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f16e1b4e7d0844879a7dd6e7cc60bf71" } modify_external_catalogs: true
I20260320 15:27:18.629544 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8567ed8118d64bda8110f902052fc1f6" } modify_external_catalogs: true
I20260320 15:27:18.629760 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8567ed8118d64bda8110f902052fc1f6" } modify_external_catalogs: true
I20260320 15:27:18.629948 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_96f581b9cc2f462faec112e20e287179" } modify_external_catalogs: true
I20260320 15:27:18.630124 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_96f581b9cc2f462faec112e20e287179" } modify_external_catalogs: true
W20260320 15:27:18.633833 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6ce1fe5280a4e888b8f1bec19ab432f has no consensus state
I20260320 15:27:18.640349 28049 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 42b7e05f45f74e5280fc47d287ae2096
W20260320 15:27:18.640764 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4346c464fb34e9dae8aeccc99dc07a3 consensus state has no leader
W20260320 15:27:18.641057 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08181eea58a24e1cb424f0e1e10f46ca consensus state has no leader
W20260320 15:27:18.648242 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db consensus state has no leader
I20260320 15:27:18.649062 28053 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 9f7198eb51774c9883419434c6cee038
I20260320 15:27:18.649453 28052 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 9393919961d04634880aadd59fbb83d2
I20260320 15:27:18.650292 28045 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 7fa8ecddb26942dba87b30d494e9652b
I20260320 15:27:18.650272 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0a7c92a1f4894df586cbb1647ed242a3
W20260320 15:27:18.652941 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 88a005b1c2b54a2fa8515f7a6b165ea8 consensus state has no leader
W20260320 15:27:18.682471 28040 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet d6ce1fe5280a4e888b8f1bec19ab432f (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet d6ce1fe5280a4e888b8f1bec19ab432f consensus state has no leader
I20260320 15:27:18.682785 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 (DEFAULT_TABLE table=default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.683022 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:18.684763 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7c92a1f4894df586cbb1647ed242a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.684899 28422 tablet_service.cc:1511] Processing CreateTablet for tablet cec531361625474f93daa8453b39f944 (DEFAULT_TABLE table=default.Table_752d88f1a66e44b49d0d3e91e2c5786c [id=715e4038b9a84544ae18e06efcf9edc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.685565 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec531361625474f93daa8453b39f944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.686913 28425 tablet_service.cc:1511] Processing CreateTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.687557 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ce1fe5280a4e888b8f1bec19ab432f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.688664 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (DEFAULT_TABLE table=default.Table_cc6df4082b8248a7bd78d01689bd8a57 [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.689100 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 8e49448a973442ad88b2090527f33899 (DEFAULT_TABLE table=default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.689265 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a005b1c2b54a2fa8515f7a6b165ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.689662 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e49448a973442ad88b2090527f33899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.689965 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 (DEFAULT_TABLE table=default.Table_11458e6c1def469191e80286da6a22c2 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.690507 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484375918aed45a9bc93e7ba5ae3a113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.690704 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 22b7f757ebdf456eb81085e5be54da4b (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.691212 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b7f757ebdf456eb81085e5be54da4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.691880 28418 tablet_service.cc:1511] Processing CreateTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a (DEFAULT_TABLE table=default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.692365 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa22d207ad8a4555aa9573b15fa1fe1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.692646 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:18.694415 28566 tablet_service.cc:1511] Processing CreateTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.695096 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (DEFAULT_TABLE table=default.Table_947a59a4bb104546b3b24ef82815ccec [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.695410 28563 tablet_service.cc:1511] Processing CreateTablet for tablet cec531361625474f93daa8453b39f944 (DEFAULT_TABLE table=default.Table_752d88f1a66e44b49d0d3e91e2c5786c [id=715e4038b9a84544ae18e06efcf9edc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.695950 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 (DEFAULT_TABLE table=default.Table_11458e6c1def469191e80286da6a22c2 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.696311 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ce1fe5280a4e888b8f1bec19ab432f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.696499 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 (DEFAULT_TABLE table=default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.697011 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec531361625474f93daa8453b39f944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.698659 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484375918aed45a9bc93e7ba5ae3a113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.699484 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a005b1c2b54a2fa8515f7a6b165ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.700320 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 22b7f757ebdf456eb81085e5be54da4b (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.700420 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 8e49448a973442ad88b2090527f33899 (DEFAULT_TABLE table=default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.700879 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b7f757ebdf456eb81085e5be54da4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.694218 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (DEFAULT_TABLE table=default.Table_84d3feb7c70543a389a534f94bd0c6ba [id=d3714568b2244ef1ab538461b3a730f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.701081 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e49448a973442ad88b2090527f33899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.701815 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7c92a1f4894df586cbb1647ed242a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.702303 28559 tablet_service.cc:1511] Processing CreateTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a (DEFAULT_TABLE table=default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.702884 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa22d207ad8a4555aa9573b15fa1fe1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.703573 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (DEFAULT_TABLE table=default.Table_84d3feb7c70543a389a534f94bd0c6ba [id=d3714568b2244ef1ab538461b3a730f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.704254 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.723308 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.746569 28733 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:18.757877 28283 tablet_service.cc:1511] Processing CreateTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 (DEFAULT_TABLE table=default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.759665 28282 tablet_service.cc:1511] Processing CreateTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.760008 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a7c92a1f4894df586cbb1647ed242a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.760766 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6ce1fe5280a4e888b8f1bec19ab432f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.760463 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (DEFAULT_TABLE table=default.Table_84d3feb7c70543a389a534f94bd0c6ba [id=d3714568b2244ef1ab538461b3a730f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.768329 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.782770 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:18.783397 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (DEFAULT_TABLE table=default.Table_cc6df4082b8248a7bd78d01689bd8a57 [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.783932 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 0a7c92a1f4894df586cbb1647ed242a3 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:18.783914 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 (DEFAULT_TABLE table=default.Table_11458e6c1def469191e80286da6a22c2 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.784061 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a005b1c2b54a2fa8515f7a6b165ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.784471 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 484375918aed45a9bc93e7ba5ae3a113. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.784865 28279 tablet_service.cc:1511] Processing CreateTablet for tablet cec531361625474f93daa8453b39f944 (DEFAULT_TABLE table=default.Table_752d88f1a66e44b49d0d3e91e2c5786c [id=715e4038b9a84544ae18e06efcf9edc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.785081 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 22b7f757ebdf456eb81085e5be54da4b from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:18.785439 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cec531361625474f93daa8453b39f944. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:18.787232 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0a7c92a1f4894df586cbb1647ed242a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a7c92a1f4894df586cbb1647ed242a3 already in progress: creating tablet
I20260320 15:27:18.787983 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 8e49448a973442ad88b2090527f33899 (DEFAULT_TABLE table=default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.788132 28734 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:18.788590 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e49448a973442ad88b2090527f33899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.789350 28733 tablet_bootstrap.cc:654] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.792065 28275 tablet_service.cc:1511] Processing CreateTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a (DEFAULT_TABLE table=default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.792856 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa22d207ad8a4555aa9573b15fa1fe1a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:18.793226 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0a7c92a1f4894df586cbb1647ed242a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a7c92a1f4894df586cbb1647ed242a3 already in progress: creating tablet
I20260320 15:27:18.796167 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 22b7f757ebdf456eb81085e5be54da4b (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.796839 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b7f757ebdf456eb81085e5be54da4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.801355 28733 log.cc:826] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:18.801672 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet d6ce1fe5280a4e888b8f1bec19ab432f from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:18.804056 28045 master_service.cc:954] ReplaceTablet: received request to replace tablet 88a005b1c2b54a2fa8515f7a6b165ea8 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:18.804612 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 88a005b1c2b54a2fa8515f7a6b165ea8 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:18.806620 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:18.807160 28048 catalog_manager.cc:6741] ReplaceTablet: tablet 22b7f757ebdf456eb81085e5be54da4b deleted and replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:18.808529 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:18.808962 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:18.809478 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22b7f757ebdf456eb81085e5be54da4b already in progress: creating tablet
W20260320 15:27:18.810051 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22b7f757ebdf456eb81085e5be54da4b already in progress: creating tablet
I20260320 15:27:18.811748 28734 tablet_bootstrap.cc:654] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.814488 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet d6ce1fe5280a4e888b8f1bec19ab432f
I20260320 15:27:18.815111 28046 catalog_manager.cc:6741] ReplaceTablet: tablet d6ce1fe5280a4e888b8f1bec19ab432f deleted and replaced by tablet dfaf72b4f25c417cafd2f53db42b3315
I20260320 15:27:18.815814 28282 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:18.815986 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:18.816527 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:18.816973 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet d6ce1fe5280a4e888b8f1bec19ab432f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6ce1fe5280a4e888b8f1bec19ab432f already in progress: creating tablet
W20260320 15:27:18.817437 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d6ce1fe5280a4e888b8f1bec19ab432f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6ce1fe5280a4e888b8f1bec19ab432f already in progress: creating tablet
I20260320 15:27:18.818645 28734 log.cc:826] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:18.820847 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c61291fb416b42aeb522d95afd859670"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.820936 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1e18f506558543ec8b4ec33dceac311d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.821638 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c61291fb416b42aeb522d95afd859670 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.821683 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e18f506558543ec8b4ec33dceac311d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.825469 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e1524b097604437f882f819270d98ea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.826143 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1524b097604437f882f819270d98ea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.829615 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:18.829542 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_53f16cd868254af4833f760b23d22b68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.830039 28045 catalog_manager.cc:6741] ReplaceTablet: tablet 88a005b1c2b54a2fa8515f7a6b165ea8 deleted and replaced by tablet e7e9568395c44af2912efb175fc1704a
W20260320 15:27:18.830142 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53f16cd868254af4833f760b23d22b68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.831383 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:18.834182 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88a005b1c2b54a2fa8515f7a6b165ea8 already in progress: creating tablet
I20260320 15:27:18.835840 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_bea154c31f5a488a950bdb13fd56705d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.836159 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_68d57b4a973849459f0df9f991026948"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.836661 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bea154c31f5a488a950bdb13fd56705d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.836612 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f36e145cb8cc41809dbbdb124102844f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.836788 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68d57b4a973849459f0df9f991026948 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.837126 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f36e145cb8cc41809dbbdb124102844f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.838105 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3c2ca98bf22746c3b0fb188339710bfa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.838591 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c2ca98bf22746c3b0fb188339710bfa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.839219 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:18.839892 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0a0f853bf916441498dce94228de6ef4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:18.840481 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a0f853bf916441498dce94228de6ef4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:18.841076 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88a005b1c2b54a2fa8515f7a6b165ea8 already in progress: creating tablet
I20260320 15:27:18.841670 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:18.845521 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_11458e6c1def469191e80286da6a22c2" } new_table_name: "default.Table_b2423a0d0aa445fb9e50e2c60c6328fa" modify_external_catalogs: true
I20260320 15:27:18.848125 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d474e70ac621425bb25bf63c5cbe3c06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:18.848770 28734 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:18.848739 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d474e70ac621425bb25bf63c5cbe3c06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:18.849107 28734 ts_tablet_manager.cc:1403] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.061s user 0.006s sys 0.005s
I20260320 15:27:18.850589 28733 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:18.850977 28733 ts_tablet_manager.cc:1403] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.105s user 0.009s sys 0.022s
I20260320 15:27:18.861407 28422 tablet_service.cc:1511] Processing CreateTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.861274 28733 raft_consensus.cc:359] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.861889 28733 raft_consensus.cc:385] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:18.861927 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2586d40343b4365ab8a67cce2e1c6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.862020 28733 raft_consensus.cc:740] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:18.863013 28733 consensus_queue.cc:260] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.865159 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2668da52d69e491eab99d1abc6909b41" } new_table_name: "default.Table_4bcf209c9cb44722b4afe02048a4898f" modify_external_catalogs: true
I20260320 15:27:18.865253 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2fdea31f3e72453d949a79a1daf3d9b6" } modify_external_catalogs: true
I20260320 15:27:18.866389 28565 tablet_service.cc:1511] Processing CreateTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.866616 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2fdea31f3e72453d949a79a1daf3d9b6" } modify_external_catalogs: true
I20260320 15:27:18.866935 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2586d40343b4365ab8a67cce2e1c6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.872303 28277 tablet_service.cc:1511] Processing CreateTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca (DEFAULT_TABLE table=default.Table_4b0fd934c3e94b18b763f7587051eacf [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.872946 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2586d40343b4365ab8a67cce2e1c6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.874019 28733 ts_tablet_manager.cc:1434] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.023s user 0.011s sys 0.009s
I20260320 15:27:18.876955 28635 heartbeater.cc:499] Master 127.27.74.61:44739 was elected leader, sending a full tablet report...
I20260320 15:27:18.880169 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_947a59a4bb104546b3b24ef82815ccec" } new_table_name: "default.Table_3b9ef310f807440f94a4042a41b0ca75" modify_external_catalogs: true
I20260320 15:27:18.882543 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4b0fd934c3e94b18b763f7587051eacf" } new_table_name: "default.Table_9c0e5be4f98e48b8aecebbf9db56e138" modify_external_catalogs: true
I20260320 15:27:18.882653 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c61291fb416b42aeb522d95afd859670" } modify_external_catalogs: true
I20260320 15:27:18.882841 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c61291fb416b42aeb522d95afd859670" } modify_external_catalogs: true
I20260320 15:27:18.883898 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_53f16cd868254af4833f760b23d22b68" } modify_external_catalogs: true
I20260320 15:27:18.884296 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_53f16cd868254af4833f760b23d22b68" } modify_external_catalogs: true
I20260320 15:27:18.885059 28733 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:18.890246 28733 tablet_bootstrap.cc:654] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.891507 28734 raft_consensus.cc:359] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.892190 28734 raft_consensus.cc:385] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:18.892376 28734 raft_consensus.cc:740] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:18.894970 28734 consensus_queue.cc:260] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.895551 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e1524b097604437f882f819270d98ea5" } modify_external_catalogs: true
I20260320 15:27:18.895794 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e1524b097604437f882f819270d98ea5" } modify_external_catalogs: true
I20260320 15:27:18.897826 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:18.899585 28733 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:18.899811 28733 ts_tablet_manager.cc:1403] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.011s
I20260320 15:27:18.901425 28733 raft_consensus.cc:359] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.901719 28733 raft_consensus.cc:385] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:18.901824 28733 raft_consensus.cc:740] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:18.902200 28733 consensus_queue.cc:260] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.902355 28734 ts_tablet_manager.cc:1434] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.053s user 0.008s sys 0.008s
I20260320 15:27:18.906436 28733 ts_tablet_manager.cc:1434] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260320 15:27:18.908885 28494 heartbeater.cc:499] Master 127.27.74.61:44739 was elected leader, sending a full tablet report...
I20260320 15:27:18.911973 28733 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:18.914770 28733 tablet_bootstrap.cc:654] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:18.919129 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4346c464fb34e9dae8aeccc99dc07a3 consensus state has no leader
I20260320 15:27:18.919646 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_752d88f1a66e44b49d0d3e91e2c5786c" } new_table_name: "default.Table_c75f89746474457f81d638cfafc1b8d5" modify_external_catalogs: true
I20260320 15:27:18.923045 28734 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:18.925431 28733 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:18.925385 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0a7c92a1f4894df586cbb1647ed242a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a7c92a1f4894df586cbb1647ed242a3 already in progress: creating tablet
I20260320 15:27:18.925690 28733 ts_tablet_manager.cc:1403] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
W20260320 15:27:18.926597 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d6ce1fe5280a4e888b8f1bec19ab432f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6ce1fe5280a4e888b8f1bec19ab432f already in progress: creating tablet
W20260320 15:27:18.926858 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88a005b1c2b54a2fa8515f7a6b165ea8 already in progress: creating tablet
I20260320 15:27:18.926749 28733 raft_consensus.cc:359] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.927031 28733 raft_consensus.cc:385] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:18.927109 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22b7f757ebdf456eb81085e5be54da4b already in progress: creating tablet
I20260320 15:27:18.927141 28733 raft_consensus.cc:740] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:18.927470 28733 consensus_queue.cc:260] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.929082 28734 tablet_bootstrap.cc:654] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.930682 28733 ts_tablet_manager.cc:1434] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:18.932880 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:18.934866 28736 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:18.935657 28495 tablet.cc:2379] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:18.937274 28565 tablet_service.cc:1511] Processing CreateTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (DEFAULT_TABLE table=default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.937261 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe4ead926da4389b131d61804451bcf (DEFAULT_TABLE table=default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.937704 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.937791 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe4ead926da4389b131d61804451bcf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:18.938414 28636 tablet.cc:2379] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:18.940524 28736 tablet_bootstrap.cc:654] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.940479 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe4ead926da4389b131d61804451bcf (DEFAULT_TABLE table=default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.940889 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe4ead926da4389b131d61804451bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.942175 28283 tablet_service.cc:1511] Processing CreateTablet for tablet 47f35263efa3413882954c8c1aa39855 (DEFAULT_TABLE table=default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.942659 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f35263efa3413882954c8c1aa39855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.942670 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 47f35263efa3413882954c8c1aa39855 (DEFAULT_TABLE table=default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.943121 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f35263efa3413882954c8c1aa39855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.943833 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 47f35263efa3413882954c8c1aa39855 (DEFAULT_TABLE table=default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.943935 28559 tablet_service.cc:1511] Processing CreateTablet for tablet e7e9568395c44af2912efb175fc1704a (DEFAULT_TABLE table=default.Table_947a59a4bb104546b3b24ef82815ccec [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.943928 28419 tablet_service.cc:1511] Processing CreateTablet for tablet e2b08bcbee8541c998a23dbc1947796a (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.944274 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f35263efa3413882954c8c1aa39855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.944304 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b08bcbee8541c998a23dbc1947796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.944343 28562 tablet_service.cc:1511] Processing CreateTablet for tablet e2b08bcbee8541c998a23dbc1947796a (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.947921 28420 tablet_service.cc:1511] Processing CreateTablet for tablet e7e9568395c44af2912efb175fc1704a (DEFAULT_TABLE table=default.Table_947a59a4bb104546b3b24ef82815ccec [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.948235 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (DEFAULT_TABLE table=default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.948382 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e9568395c44af2912efb175fc1704a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.948626 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.950488 28733 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:18.951840 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b08bcbee8541c998a23dbc1947796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.952031 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 1fe4ead926da4389b131d61804451bcf (DEFAULT_TABLE table=default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.952337 28733 tablet_bootstrap.cc:654] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:18.952591 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fe4ead926da4389b131d61804451bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.958560 28563 tablet_service.cc:1511] Processing CreateTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.958858 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.62:43951
--webserver_interface=127.27.74.62
--webserver_port=38201
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260320 15:27:18.959095 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfaf72b4f25c417cafd2f53db42b3315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.959267 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (DEFAULT_TABLE table=default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.959499 28277 tablet_service.cc:1511] Processing CreateTablet for tablet e2b08bcbee8541c998a23dbc1947796a (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.959733 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.959959 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2b08bcbee8541c998a23dbc1947796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.963887 28276 tablet_service.cc:1511] Processing CreateTablet for tablet e7e9568395c44af2912efb175fc1704a (DEFAULT_TABLE table=default.Table_947a59a4bb104546b3b24ef82815ccec [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.964447 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e9568395c44af2912efb175fc1704a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.967080 28279 tablet_service.cc:1511] Processing CreateTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.967585 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfaf72b4f25c417cafd2f53db42b3315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.968959 28420 tablet_service.cc:1511] Processing CreateTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:18.973714 28736 log.cc:826] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:18.974169 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfaf72b4f25c417cafd2f53db42b3315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.974323 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7e9568395c44af2912efb175fc1704a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:18.984608 28733 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:18.984892 28733 ts_tablet_manager.cc:1403] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.007s
I20260320 15:27:18.985814 28733 raft_consensus.cc:359] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:18.987670 28733 raft_consensus.cc:385] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:18.987857 28733 raft_consensus.cc:740] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:18.991621 28733 consensus_queue.cc:260] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.000691 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 1fe4ead926da4389b131d61804451bcf
I20260320 15:27:19.001408 28733 ts_tablet_manager.cc:1434] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260320 15:27:19.001586 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_68d57b4a973849459f0df9f991026948" } modify_external_catalogs: true
I20260320 15:27:19.001773 28733 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.001792 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_68d57b4a973849459f0df9f991026948" } modify_external_catalogs: true
I20260320 15:27:19.003715 28733 tablet_bootstrap.cc:654] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.003832 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.004244 28734 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:19.004665 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 1fe4ead926da4389b131d61804451bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fe4ead926da4389b131d61804451bcf already in progress: creating tablet
I20260320 15:27:19.005291 28734 ts_tablet_manager.cc:1403] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.094s user 0.000s sys 0.004s
I20260320 15:27:19.005968 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.006502 28734 raft_consensus.cc:359] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.006783 28734 raft_consensus.cc:385] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:19.007128 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1fe4ead926da4389b131d61804451bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fe4ead926da4389b131d61804451bcf already in progress: creating tablet
I20260320 15:27:19.006953 28734 raft_consensus.cc:740] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.007688 28734 consensus_queue.cc:260] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.009867 28734 ts_tablet_manager.cc:1434] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260320 15:27:19.010468 28734 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.012471 28734 tablet_bootstrap.cc:654] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.016528 28733 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.016777 28733 ts_tablet_manager.cc:1403] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260320 15:27:19.017654 28733 raft_consensus.cc:359] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.017889 28733 raft_consensus.cc:385] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.017958 28733 raft_consensus.cc:740] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.019694 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 4902ffdcdbc54d5aa9c71f6cd239c69a
I20260320 15:27:19.019853 28733 consensus_queue.cc:260] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.020028 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 47f35263efa3413882954c8c1aa39855
I20260320 15:27:19.020427 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 88a005b1c2b54a2fa8515f7a6b165ea8 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.020547 28733 ts_tablet_manager.cc:1434] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260320 15:27:19.020642 28054 master_service.cc:954] ReplaceTablet: received request to replace tablet e2b08bcbee8541c998a23dbc1947796a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.020901 28733 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.021462 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.021694 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.022713 28045 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.022838 28733 tablet_bootstrap.cc:654] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:19.024143 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7e9568395c44af2912efb175fc1704a consensus state has no leader
W20260320 15:27:19.024559 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cec531361625474f93daa8453b39f944 consensus state has no leader
I20260320 15:27:19.024835 28734 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:19.024881 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 484375918aed45a9bc93e7ba5ae3a113 consensus state has no leader
I20260320 15:27:19.025038 28734 ts_tablet_manager.cc:1403] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.003s
W20260320 15:27:19.026047 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e2586d40343b4365ab8a67cce2e1c6ca consensus state has no leader
I20260320 15:27:19.026475 28734 raft_consensus.cc:359] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.026706 28734 raft_consensus.cc:385] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.026767 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:19.026799 28734 raft_consensus.cc:740] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.027045 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.027102 28734 consensus_queue.cc:260] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.027683 28734 ts_tablet_manager.cc:1434] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260320 15:27:19.028285 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.028481 28734 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.028672 28729 tablet_replica.cc:333] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:19.029232 28729 raft_consensus.cc:2243] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.029717 28729 raft_consensus.cc:2272] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:19.030675 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 47f35263efa3413882954c8c1aa39855 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47f35263efa3413882954c8c1aa39855 already in progress: creating tablet
I20260320 15:27:19.031239 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.031654 28729 ts_tablet_manager.cc:1916] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.031946 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:19.032781 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fa22d207ad8a4555aa9573b15fa1fe1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa22d207ad8a4555aa9573b15fa1fe1a already in progress: creating tablet
W20260320 15:27:19.033025 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902ffdcdbc54d5aa9c71f6cd239c69a already in progress: creating tablet
I20260320 15:27:19.033325 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.033624 28739 raft_consensus.cc:493] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.033877 28739 raft_consensus.cc:515] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.036039 28739 leader_election.cc:290] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.045945 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.046957 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.047595 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.048256 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.050567 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_29bb7e2e67c848a19e0c4a4e712d4788"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.051201 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29bb7e2e67c848a19e0c4a4e712d4788 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.052037 28729 ts_tablet_manager.cc:1929] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.052613 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_cc75ea0f87d444678d04cdc9dcaa4a1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.053081 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 47f35263efa3413882954c8c1aa39855 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47f35263efa3413882954c8c1aa39855 already in progress: creating tablet
W20260320 15:27:19.053375 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902ffdcdbc54d5aa9c71f6cd239c69a already in progress: creating tablet
W20260320 15:27:19.053545 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc75ea0f87d444678d04cdc9dcaa4a1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.054662 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet fa22d207ad8a4555aa9573b15fa1fe1a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.055187 28049 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 7a9de54cae73466ea4bdc35cb0b6f63d
I20260320 15:27:19.061043 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.061815 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.062016 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.062229 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.062424 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.062606 28730 tablet_replica.cc:333] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.063094 28730 raft_consensus.cc:2243] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.063503 28730 raft_consensus.cc:2272] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.064903 28049 master_service.cc:954] ReplaceTablet: received request to replace tablet e7e9568395c44af2912efb175fc1704a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.065003 28739 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.065239 28739 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.065917 28730 ts_tablet_manager.cc:1916] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.066181 28739 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.066960 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet 0a7c92a1f4894df586cbb1647ed242a3 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.069023 28736 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:19.069415 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c4feb7323b9748c9b7f01574beecc884"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.070102 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4feb7323b9748c9b7f01574beecc884 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.071976 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_bd4efcc6628b40ddbd5b20ee45ec947b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.072544 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd4efcc6628b40ddbd5b20ee45ec947b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.073168 28729 log.cc:1199] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:19.073701 28729 ts_tablet_manager.cc:1950] T fa22d207ad8a4555aa9573b15fa1fe1a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.075811 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet fa22d207ad8a4555aa9573b15fa1fe1a (table default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]) successfully deleted
I20260320 15:27:19.076545 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet e2b08bcbee8541c998a23dbc1947796a
I20260320 15:27:19.077025 28734 tablet_bootstrap.cc:654] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.077083 28054 catalog_manager.cc:6741] ReplaceTablet: tablet e2b08bcbee8541c998a23dbc1947796a deleted and replaced by tablet 0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:19.078611 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.079003 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.077919 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:19.080636 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e2b08bcbee8541c998a23dbc1947796a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2b08bcbee8541c998a23dbc1947796a already in progress: creating tablet
I20260320 15:27:19.081645 28730 ts_tablet_manager.cc:1929] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.081698 28736 ts_tablet_manager.cc:1403] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.147s user 0.012s sys 0.000s
I20260320 15:27:19.083410 28730 log.cc:1199] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0a7c92a1f4894df586cbb1647ed242a3
I20260320 15:27:19.068893 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.069275 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.085155 28730 ts_tablet_manager.cc:1950] T 0a7c92a1f4894df586cbb1647ed242a3 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.086032 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_adca08e7f2314aec8ad43dc91db184de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.086731 28729 tablet_replica.cc:333] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:19.086803 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adca08e7f2314aec8ad43dc91db184de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.086975 28729 raft_consensus.cc:2243] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.087080 28734 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.087194 28729 raft_consensus.cc:2272] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.087271 28734 ts_tablet_manager.cc:1403] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.001s
I20260320 15:27:19.089562 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 0a7c92a1f4894df586cbb1647ed242a3 (table default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]) successfully deleted
I20260320 15:27:19.090118 28729 ts_tablet_manager.cc:1916] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.090169 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_55de4adf4e2242bf959513e74510cb9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.090453 28734 raft_consensus.cc:359] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.090667 28734 raft_consensus.cc:385] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.090761 28734 raft_consensus.cc:740] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:19.090909 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55de4adf4e2242bf959513e74510cb9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.090930 28741 raft_consensus.cc:493] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.091010 28734 consensus_queue.cc:260] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.091152 28741 raft_consensus.cc:515] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.091665 28730 tablet_replica.cc:333] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.091883 28730 raft_consensus.cc:2243] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.092065 28730 raft_consensus.cc:2272] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.093428 28741 leader_election.cc:290] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.094756 28730 ts_tablet_manager.cc:1916] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.098579 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_15f5a33964a94937ada1a7df25fda363"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.099457 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15f5a33964a94937ada1a7df25fda363 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.100385 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1832ad9fbd5449c9bafcf89adc062c0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.101289 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1832ad9fbd5449c9bafcf89adc062c0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:19.102391 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 1fe4ead926da4389b131d61804451bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fe4ead926da4389b131d61804451bcf already in progress: creating tablet
I20260320 15:27:19.110723 28736 raft_consensus.cc:359] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.121376 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet e7e9568395c44af2912efb175fc1704a
I20260320 15:27:19.121654 28730 ts_tablet_manager.cc:1929] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.121819 28730 log.cc:1199] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:19.121978 28049 catalog_manager.cc:6741] ReplaceTablet: tablet e7e9568395c44af2912efb175fc1704a deleted and replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:19.122193 28730 ts_tablet_manager.cc:1950] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.122886 28730 tablet_replica.cc:333] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.123045 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.123126 28730 raft_consensus.cc:2243] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.123353 28730 raft_consensus.cc:2272] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.122309 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.124308 28736 raft_consensus.cc:385] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.124475 28736 raft_consensus.cc:740] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:19.124483 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e7e9568395c44af2912efb175fc1704a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7e9568395c44af2912efb175fc1704a already in progress: creating tablet
I20260320 15:27:19.125578 28736 consensus_queue.cc:260] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.126953 28730 ts_tablet_manager.cc:1916] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.128937 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.130808 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a005b1c2b54a2fa8515f7a6b165ea8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.131346 28447 raft_consensus.cc:2468] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:19.131881 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (table default.Table_3b9ef310f807440f94a4042a41b0ca75 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:19.132550 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b7f757ebdf456eb81085e5be54da4b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.133338 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e5a27bce7ad246d8807536e43ccd8b2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.134058 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e5a27bce7ad246d8807536e43ccd8b2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.134227 28524 leader_election.cc:304] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:19.134176 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a005b1c2b54a2fa8515f7a6b165ea8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:19.134644 28524 raft_consensus.cc:2686] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260320 15:27:19.134939 28524 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.135238 28524 leader_election.cc:343] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.135511 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b7f757ebdf456eb81085e5be54da4b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:19.137102 28524 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.138855 28524 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:19.139581 28524 raft_consensus.cc:2686] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:19.138217 28730 ts_tablet_manager.cc:1929] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.138644 28733 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.141197 28733 ts_tablet_manager.cc:1403] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.120s user 0.005s sys 0.004s
I20260320 15:27:19.142458 28733 raft_consensus.cc:359] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.142848 28733 raft_consensus.cc:385] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.143071 28733 raft_consensus.cc:740] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.143491 28733 consensus_queue.cc:260] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.144166 28733 ts_tablet_manager.cc:1434] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:19.144724 28733 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.146780 28733 tablet_bootstrap.cc:654] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.144906 28730 log.cc:1199] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/22b7f757ebdf456eb81085e5be54da4b
W20260320 15:27:19.149835 28382 leader_election.cc:343] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:19.150211 28730 ts_tablet_manager.cc:1950] T 22b7f757ebdf456eb81085e5be54da4b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.153023 28734 ts_tablet_manager.cc:1434] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.066s user 0.008s sys 0.019s
I20260320 15:27:19.153468 28734 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.155927 28734 tablet_bootstrap.cc:654] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.156687 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a005b1c2b54a2fa8515f7a6b165ea8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.157303 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 22b7f757ebdf456eb81085e5be54da4b (table default.Table_9c0e5be4f98e48b8aecebbf9db56e138 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
W20260320 15:27:19.157632 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet fa22d207ad8a4555aa9573b15fa1fe1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa22d207ad8a4555aa9573b15fa1fe1a already in progress: creating tablet
W20260320 15:27:19.157856 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet d6ce1fe5280a4e888b8f1bec19ab432f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6ce1fe5280a4e888b8f1bec19ab432f already in progress: creating tablet
W20260320 15:27:19.158079 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet e2b08bcbee8541c998a23dbc1947796a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2b08bcbee8541c998a23dbc1947796a already in progress: creating tablet
W20260320 15:27:19.158325 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet e7e9568395c44af2912efb175fc1704a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7e9568395c44af2912efb175fc1704a already in progress: creating tablet
W20260320 15:27:19.158311 28383 leader_election.cc:343] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.158560 28383 leader_election.cc:304] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:19.159416 28753 raft_consensus.cc:2749] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.160454 28351 heartbeater.cc:499] Master 127.27.74.61:44739 was elected leader, sending a full tablet report...
I20260320 15:27:19.164232 28734 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.164439 28734 ts_tablet_manager.cc:1403] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260320 15:27:19.164294 28736 ts_tablet_manager.cc:1434] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.082s user 0.007s sys 0.016s
I20260320 15:27:19.165282 28734 raft_consensus.cc:359] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.165557 28734 raft_consensus.cc:385] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.165647 28734 raft_consensus.cc:740] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.166381 28729 ts_tablet_manager.cc:1929] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.166648 28729 log.cc:1199] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/d6ce1fe5280a4e888b8f1bec19ab432f
I20260320 15:27:19.167305 28736 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:19.168289 28729 ts_tablet_manager.cc:1950] T d6ce1fe5280a4e888b8f1bec19ab432f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.169559 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet d6ce1fe5280a4e888b8f1bec19ab432f (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
I20260320 15:27:19.170131 28736 tablet_bootstrap.cc:654] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:19.170552 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0a7c92a1f4894df586cbb1647ed242a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a7c92a1f4894df586cbb1647ed242a3 already in progress: creating tablet
I20260320 15:27:19.170506 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_307076d582aa44b9be757a3307398132"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.171062 28729 tablet_replica.cc:333] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:19.171267 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_307076d582aa44b9be757a3307398132 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.172353 28733 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.172540 28733 ts_tablet_manager.cc:1403] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.005s
I20260320 15:27:19.173395 28733 raft_consensus.cc:359] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.173629 28733 raft_consensus.cc:385] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.173713 28733 raft_consensus.cc:740] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.174022 28733 consensus_queue.cc:260] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.165908 28734 consensus_queue.cc:260] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.176079 28734 ts_tablet_manager.cc:1434] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260320 15:27:19.176509 28734 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.179054 28733 ts_tablet_manager.cc:1434] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260320 15:27:19.179127 28734 tablet_bootstrap.cc:654] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.180229 28733 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.182222 28733 tablet_bootstrap.cc:654] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.171312 28729 raft_consensus.cc:2243] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.183368 28729 raft_consensus.cc:2272] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.184440 28729 ts_tablet_manager.cc:1916] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.192960 28729 ts_tablet_manager.cc:1929] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.194803 28733 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.195130 28733 ts_tablet_manager.cc:1403] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.002s
I20260320 15:27:19.196135 28733 raft_consensus.cc:359] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.196434 28733 raft_consensus.cc:385] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.196576 28733 raft_consensus.cc:740] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.197008 28733 consensus_queue.cc:260] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.197825 28733 ts_tablet_manager.cc:1434] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:19.198238 28733 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.199872 28736 tablet_bootstrap.cc:492] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:19.200145 28736 ts_tablet_manager.cc:1403] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.012s
I20260320 15:27:19.201447 28736 raft_consensus.cc:359] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.201779 28736 raft_consensus.cc:385] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.201956 28736 raft_consensus.cc:740] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.202558 28736 consensus_queue.cc:260] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.205768 28736 ts_tablet_manager.cc:1434] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:19.207944 28733 tablet_bootstrap.cc:654] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.207850 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bea154c31f5a488a950bdb13fd56705d" } new_table_name: "default.Table_945b0cecf5834f899f177152b6153d2d" modify_external_catalogs: true
I20260320 15:27:19.209498 28736 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:19.211450 28736 tablet_bootstrap.cc:654] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.211761 28729 log.cc:1199] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:19.212321 28729 ts_tablet_manager.cc:1950] T 88a005b1c2b54a2fa8515f7a6b165ea8 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.214692 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (table default.Table_3b9ef310f807440f94a4042a41b0ca75 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:19.215865 28734 tablet_bootstrap.cc:492] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.216607 28733 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:19.216593 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22b7f757ebdf456eb81085e5be54da4b already in progress: creating tablet
I20260320 15:27:19.216799 28733 ts_tablet_manager.cc:1403] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
W20260320 15:27:19.217497 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e2b08bcbee8541c998a23dbc1947796a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2b08bcbee8541c998a23dbc1947796a already in progress: creating tablet
I20260320 15:27:19.217587 28733 raft_consensus.cc:359] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.217840 28733 raft_consensus.cc:385] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.217932 28733 raft_consensus.cc:740] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.218191 28733 consensus_queue.cc:260] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.218715 28733 ts_tablet_manager.cc:1434] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260320 15:27:19.218739 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 47f35263efa3413882954c8c1aa39855 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47f35263efa3413882954c8c1aa39855 already in progress: creating tablet
I20260320 15:27:19.219043 28733 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.219053 28734 ts_tablet_manager.cc:1403] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.043s user 0.013s sys 0.008s
I20260320 15:27:19.220099 28734 raft_consensus.cc:359] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.220845 28733 tablet_bootstrap.cc:654] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:19.220891 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902ffdcdbc54d5aa9c71f6cd239c69a already in progress: creating tablet
W20260320 15:27:19.221658 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e7e9568395c44af2912efb175fc1704a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7e9568395c44af2912efb175fc1704a already in progress: creating tablet
I20260320 15:27:19.222016 28734 raft_consensus.cc:385] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.222191 28734 raft_consensus.cc:740] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.222569 28734 consensus_queue.cc:260] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.223321 28734 ts_tablet_manager.cc:1434] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:19.223785 28734 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.225447 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c75f89746474457f81d638cfafc1b8d5" } modify_external_catalogs: true
I20260320 15:27:19.225647 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c75f89746474457f81d638cfafc1b8d5" } modify_external_catalogs: true
I20260320 15:27:19.227248 28734 tablet_bootstrap.cc:654] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.232095 28733 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.232301 28733 ts_tablet_manager.cc:1403] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260320 15:27:19.233248 28733 raft_consensus.cc:359] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.233512 28733 raft_consensus.cc:385] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.233623 28733 raft_consensus.cc:740] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.233884 28733 consensus_queue.cc:260] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.234426 28733 ts_tablet_manager.cc:1434] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:19.234714 28733 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.237772 28733 tablet_bootstrap.cc:654] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.242483 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0a0f853bf916441498dce94228de6ef4" } new_table_name: "default.Table_66d9e5b794ae4f1298e4264693af9a3c" modify_external_catalogs: true
I20260320 15:27:19.244376 28734 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.244690 28734 ts_tablet_manager.cc:1403] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.004s
I20260320 15:27:19.245658 28734 raft_consensus.cc:359] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.245973 28734 raft_consensus.cc:385] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.246111 28734 raft_consensus.cc:740] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.246456 28734 consensus_queue.cc:260] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.247073 28734 ts_tablet_manager.cc:1434] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:19.247478 28734 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.249492 28734 tablet_bootstrap.cc:654] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.252720 28733 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.252941 28733 ts_tablet_manager.cc:1403] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.006s
I20260320 15:27:19.254160 28733 raft_consensus.cc:359] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.254402 28733 raft_consensus.cc:385] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.254487 28733 raft_consensus.cc:740] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.254786 28733 consensus_queue.cc:260] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.255101 28751 raft_consensus.cc:493] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.255345 28751 raft_consensus.cc:515] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.256327 28751 leader_election.cc:290] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.257671 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe4ead926da4389b131d61804451bcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.257907 28733 ts_tablet_manager.cc:1434] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:19.258176 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe4ead926da4389b131d61804451bcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.258322 28733 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:19.259992 28524 leader_election.cc:343] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.260263 28524 leader_election.cc:343] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.260385 28524 leader_election.cc:304] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:19.261176 28751 raft_consensus.cc:2749] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.261474 28733 tablet_bootstrap.cc:654] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.263455 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_05ab9b1188db49fdb97c2cd0f3acadbe" } modify_external_catalogs: true
I20260320 15:27:19.263731 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_05ab9b1188db49fdb97c2cd0f3acadbe" } modify_external_catalogs: true
I20260320 15:27:19.268899 28734 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.268999 28733 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.269168 28733 ts_tablet_manager.cc:1403] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260320 15:27:19.269215 28734 ts_tablet_manager.cc:1403] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.011s
I20260320 15:27:19.270078 28733 raft_consensus.cc:359] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.270290 28733 raft_consensus.cc:385] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.270385 28733 raft_consensus.cc:740] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.270190 28734 raft_consensus.cc:359] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.270658 28734 raft_consensus.cc:385] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.270670 28733 consensus_queue.cc:260] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.271216 28733 ts_tablet_manager.cc:1434] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:19.271524 28733 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.270800 28734 raft_consensus.cc:740] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.273120 28734 consensus_queue.cc:260] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.276024 28734 ts_tablet_manager.cc:1434] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:19.276484 28734 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.278846 28045 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.281709 28733 tablet_bootstrap.cc:654] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.282008 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee (DEFAULT_TABLE table=default.Table_d474e70ac621425bb25bf63c5cbe3c06 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.282573 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 801b3b0ba2344cbe8d89111e17e546ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.283432 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet cec531361625474f93daa8453b39f944 (table default.Table_c75f89746474457f81d638cfafc1b8d5 [id=715e4038b9a84544ae18e06efcf9edc9]). Expected version 1 got 0
I20260320 15:27:19.287052 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_b2423a0d0aa445fb9e50e2c60c6328fa [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 1 got 0
I20260320 15:27:19.287287 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.290306 28733 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.290501 28733 ts_tablet_manager.cc:1403] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.006s
I20260320 15:27:19.290998 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 026bb078f3904a44820442e69e654f72 (DEFAULT_TABLE table=default.Table_f36e145cb8cc41809dbbdb124102844f [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.291303 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_9c0e5be4f98e48b8aecebbf9db56e138 [id=0466f754fc5f43249331fda298bb726c]). Expected version 1 got 0
I20260320 15:27:19.291342 28283 tablet_service.cc:1511] Processing CreateTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee (DEFAULT_TABLE table=default.Table_d474e70ac621425bb25bf63c5cbe3c06 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.291654 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bb078f3904a44820442e69e654f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.291400 28733 raft_consensus.cc:359] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.291790 28733 raft_consensus.cc:385] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.291883 28733 raft_consensus.cc:740] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.291989 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet cec531361625474f93daa8453b39f944 (table default.Table_c75f89746474457f81d638cfafc1b8d5 [id=715e4038b9a84544ae18e06efcf9edc9]). Expected version 1 got 0
I20260320 15:27:19.292140 28733 consensus_queue.cc:260] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.292253 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet dfaf72b4f25c417cafd2f53db42b3315 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]). Expected version 2 got 0
I20260320 15:27:19.292356 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 801b3b0ba2344cbe8d89111e17e546ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.292692 28733 ts_tablet_manager.cc:1434] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:19.294376 28733 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.296146 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (DEFAULT_TABLE table=default.Table_0a0f853bf916441498dce94228de6ef4 [id=d9929a996d914fb387ec71330be86b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.296767 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351a4cc8692a472d9bc5f3dbcc5ee172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.296180 28279 tablet_service.cc:1511] Processing CreateTablet for tablet df5ffaedeaf94def93d93a538c42e3bd (DEFAULT_TABLE table=default.Table_bea154c31f5a488a950bdb13fd56705d [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.297138 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5ffaedeaf94def93d93a538c42e3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.297224 28733 tablet_bootstrap.cc:654] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.297655 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 (DEFAULT_TABLE table=default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.298056 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3284c73cfa4dfba65a88a7dd243190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.306697 28420 tablet_service.cc:1511] Processing CreateTablet for tablet df5ffaedeaf94def93d93a538c42e3bd (DEFAULT_TABLE table=default.Table_bea154c31f5a488a950bdb13fd56705d [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.306944 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee (DEFAULT_TABLE table=default.Table_d474e70ac621425bb25bf63c5cbe3c06 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.307406 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5ffaedeaf94def93d93a538c42e3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.307790 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (DEFAULT_TABLE table=default.Table_0a0f853bf916441498dce94228de6ef4 [id=d9929a996d914fb387ec71330be86b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.308228 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351a4cc8692a472d9bc5f3dbcc5ee172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.309250 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cc75ea0f87d444678d04cdc9dcaa4a1c" } modify_external_catalogs: true
I20260320 15:27:19.309494 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cc75ea0f87d444678d04cdc9dcaa4a1c" } modify_external_catalogs: true
I20260320 15:27:19.312752 28736 tablet_bootstrap.cc:492] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:19.312995 28736 ts_tablet_manager.cc:1403] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.104s user 0.003s sys 0.003s
I20260320 15:27:19.313905 28736 raft_consensus.cc:359] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.314116 28736 raft_consensus.cc:385] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.314210 28736 raft_consensus.cc:740] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.314601 28736 consensus_queue.cc:260] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.317246 28736 ts_tablet_manager.cc:1434] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260320 15:27:19.320183 28736 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:19.322316 28736 tablet_bootstrap.cc:654] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.322551 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 (DEFAULT_TABLE table=default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.324254 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bd4efcc6628b40ddbd5b20ee45ec947b" } new_table_name: "default.Table_2cf33331acf14700b31b2edd8666f285" modify_external_catalogs: true
I20260320 15:27:19.330349 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 026bb078f3904a44820442e69e654f72 (DEFAULT_TABLE table=default.Table_f36e145cb8cc41809dbbdb124102844f [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.330969 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 801b3b0ba2344cbe8d89111e17e546ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.336381 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 026bb078f3904a44820442e69e654f72 (DEFAULT_TABLE table=default.Table_f36e145cb8cc41809dbbdb124102844f [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.337033 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bb078f3904a44820442e69e654f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.337204 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 (DEFAULT_TABLE table=default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.337641 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3284c73cfa4dfba65a88a7dd243190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.339453 28563 tablet_service.cc:1511] Processing CreateTablet for tablet df5ffaedeaf94def93d93a538c42e3bd (DEFAULT_TABLE table=default.Table_bea154c31f5a488a950bdb13fd56705d [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.340006 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5ffaedeaf94def93d93a538c42e3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.341825 28734 tablet_bootstrap.cc:654] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.343397 28751 raft_consensus.cc:493] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.343605 28751 raft_consensus.cc:515] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.343801 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (DEFAULT_TABLE table=default.Table_0a0f853bf916441498dce94228de6ef4 [id=d9929a996d914fb387ec71330be86b22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.344385 28751 leader_election.cc:290] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.344422 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 351a4cc8692a472d9bc5f3dbcc5ee172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.344713 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bb078f3904a44820442e69e654f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.346204 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4902ffdcdbc54d5aa9c71f6cd239c69a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:19.347298 28524 leader_election.cc:343] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.348157 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4902ffdcdbc54d5aa9c71f6cd239c69a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:19.348973 28524 leader_election.cc:343] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.349300 28524 leader_election.cc:304] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:19.349850 28751 raft_consensus.cc:2749] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.350066 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3284c73cfa4dfba65a88a7dd243190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.371076 28733 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.371285 28733 ts_tablet_manager.cc:1403] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.077s user 0.019s sys 0.054s
I20260320 15:27:19.372239 28733 raft_consensus.cc:359] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.372460 28733 raft_consensus.cc:385] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.372565 28733 raft_consensus.cc:740] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.372852 28733 consensus_queue.cc:260] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.373373 28733 ts_tablet_manager.cc:1434] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:19.373802 28733 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.376242 28736 tablet_bootstrap.cc:492] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:19.376554 28736 ts_tablet_manager.cc:1403] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.056s user 0.005s sys 0.013s
I20260320 15:27:19.376585 28733 tablet_bootstrap.cc:654] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.377496 28736 raft_consensus.cc:359] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.377735 28736 raft_consensus.cc:385] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.377832 28736 raft_consensus.cc:740] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.378170 28736 consensus_queue.cc:260] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.385035 28733 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.385244 28733 ts_tablet_manager.cc:1403] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260320 15:27:19.386041 28733 raft_consensus.cc:359] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.386271 28733 raft_consensus.cc:385] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.386626 28733 raft_consensus.cc:740] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:19.386930 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df5ffaedeaf94def93d93a538c42e3bd consensus state has no leader
I20260320 15:27:19.386940 28733 consensus_queue.cc:260] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:19.387069 28352 tablet.cc:2379] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:19.387487 28733 ts_tablet_manager.cc:1434] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260320 15:27:19.387631 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 351a4cc8692a472d9bc5f3dbcc5ee172 consensus state has no leader
I20260320 15:27:19.390110 28733 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.391590 28756 raft_consensus.cc:493] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.391846 28756 raft_consensus.cc:515] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.392123 28733 tablet_bootstrap.cc:654] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.393846 28756 leader_election.cc:290] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.394383 28733 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.394567 28733 ts_tablet_manager.cc:1403] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:19.395599 28733 raft_consensus.cc:359] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.395833 28733 raft_consensus.cc:385] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.395844 28756 raft_consensus.cc:493] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.395941 28733 raft_consensus.cc:740] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.396032 28756 raft_consensus.cc:515] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.396266 28733 consensus_queue.cc:260] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.396768 28756 leader_election.cc:290] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.396965 28733 ts_tablet_manager.cc:1434] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:19.397359 28733 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.399459 28733 tablet_bootstrap.cc:654] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.402055 28733 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.402272 28733 ts_tablet_manager.cc:1403] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260320 15:27:19.403245 28733 raft_consensus.cc:359] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.403517 28733 raft_consensus.cc:385] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.403631 28733 raft_consensus.cc:740] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.403949 28733 consensus_queue.cc:260] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.404577 28733 ts_tablet_manager.cc:1434] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:19.404958 28733 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.404887 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef9ab0a1f984e528f0fd7f7ce7a79db" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:19.405089 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ce1fe5280a4e888b8f1bec19ab432f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:19.405361 28588 raft_consensus.cc:2468] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:19.405445 28587 raft_consensus.cc:2468] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:19.405725 28734 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.405905 28734 ts_tablet_manager.cc:1403] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.129s user 0.001s sys 0.004s
I20260320 15:27:19.406960 28733 tablet_bootstrap.cc:654] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.406932 28734 raft_consensus.cc:359] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.407269 28734 raft_consensus.cc:385] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.407369 28734 raft_consensus.cc:740] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.407955 28734 consensus_queue.cc:260] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.408169 28239 leader_election.cc:304] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:19.408711 28239 leader_election.cc:304] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:19.408753 28734 ts_tablet_manager.cc:1434] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260320 15:27:19.409117 28734 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.409145 28756 raft_consensus.cc:2804] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.409291 28756 raft_consensus.cc:493] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.409430 28756 raft_consensus.cc:3060] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.411633 28734 tablet_bootstrap.cc:654] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.417290 28733 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.417536 28733 ts_tablet_manager.cc:1403] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260320 15:27:19.418931 28756 raft_consensus.cc:515] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.419677 28733 raft_consensus.cc:359] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.419893 28733 raft_consensus.cc:385] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.419997 28733 raft_consensus.cc:740] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.420508 28733 consensus_queue.cc:260] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.421170 28733 ts_tablet_manager.cc:1434] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:19.421519 28733 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.422013 28734 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.422223 28734 ts_tablet_manager.cc:1403] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260320 15:27:19.423652 28733 tablet_bootstrap.cc:654] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.423559 28734 raft_consensus.cc:359] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.423818 28734 raft_consensus.cc:385] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.423942 28734 raft_consensus.cc:740] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.424283 28779 raft_consensus.cc:2804] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.424286 28734 consensus_queue.cc:260] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.424475 28779 raft_consensus.cc:493] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.424757 28779 raft_consensus.cc:3060] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.428923 28734 ts_tablet_manager.cc:1434] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:19.428949 28779 raft_consensus.cc:515] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.430126 28779 leader_election.cc:290] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.431085 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6ce1fe5280a4e888b8f1bec19ab432f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:19.431357 28587 raft_consensus.cc:3060] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.432615 28734 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.434978 28734 tablet_bootstrap.cc:654] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.435182 28587 raft_consensus.cc:2468] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:19.435364 28733 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.435601 28733 ts_tablet_manager.cc:1403] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260320 15:27:19.436594 28733 raft_consensus.cc:359] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.436844 28733 raft_consensus.cc:385] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.436930 28733 raft_consensus.cc:740] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.436883 28239 leader_election.cc:304] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:19.437237 28733 consensus_queue.cc:260] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.437762 28733 ts_tablet_manager.cc:1434] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:19.437821 28779 raft_consensus.cc:2804] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:19.438129 28733 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.441627 28733 tablet_bootstrap.cc:654] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.442245 28756 leader_election.cc:290] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.443974 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef9ab0a1f984e528f0fd7f7ce7a79db" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:19.444344 28587 raft_consensus.cc:3060] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.450083 28733 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.450258 28733 ts_tablet_manager.cc:1403] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260320 15:27:19.452559 28733 raft_consensus.cc:359] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.452826 28733 raft_consensus.cc:385] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.452942 28733 raft_consensus.cc:740] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.453284 28733 consensus_queue.cc:260] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.454698 28733 ts_tablet_manager.cc:1434] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260320 15:27:19.455221 28733 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.456329 28779 raft_consensus.cc:697] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:19.457382 28733 tablet_bootstrap.cc:654] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.457362 28779 consensus_queue.cc:237] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.461609 28587 raft_consensus.cc:2468] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:19.462347 28239 leader_election.cc:304] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:19.467176 28733 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.467428 28733 ts_tablet_manager.cc:1403] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260320 15:27:19.468665 28733 raft_consensus.cc:359] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.468982 28733 raft_consensus.cc:385] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.469074 28733 raft_consensus.cc:740] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.469470 28733 consensus_queue.cc:260] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.470220 28733 ts_tablet_manager.cc:1434] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:19.474017 28779 raft_consensus.cc:2804] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:19.474306 28779 raft_consensus.cc:697] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:19.474756 28779 consensus_queue.cc:237] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:19.477281 28736 ts_tablet_manager.cc:1434] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.101s user 0.000s sys 0.003s
I20260320 15:27:19.477757 28736 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:19.492412 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 08181eea58a24e1cb424f0e1e10f46ca
I20260320 15:27:19.493177 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet e2b08bcbee8541c998a23dbc1947796a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.493603 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet cec531361625474f93daa8453b39f944
I20260320 15:27:19.496361 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 1fe4ead926da4389b131d61804451bcf from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:19.498597 28046 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet dfaf72b4f25c417cafd2f53db42b3315 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet dfaf72b4f25c417cafd2f53db42b3315 consensus state has no leader
I20260320 15:27:19.499258 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet 47f35263efa3413882954c8c1aa39855 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.504891 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 08181eea58a24e1cb424f0e1e10f46ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.504890 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_29bb7e2e67c848a19e0c4a4e712d4788" } new_table_name: "default.Table_0149a83e663749ef9b7aaf5e635463d5" modify_external_catalogs: true
I20260320 15:27:19.505213 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet cec531361625474f93daa8453b39f944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.505832 28729 tablet_replica.cc:333] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:19.506042 28729 raft_consensus.cc:2243] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.506234 28729 raft_consensus.cc:2272] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.506546 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3b9ef310f807440f94a4042a41b0ca75" } new_table_name: "default.Table_0c2bc058015240478a72fe93a91a9727" modify_external_catalogs: true
I20260320 15:27:19.508708 28729 ts_tablet_manager.cc:1916] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.511823 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3c2ca98bf22746c3b0fb188339710bfa" } modify_external_catalogs: true
I20260320 15:27:19.512012 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3c2ca98bf22746c3b0fb188339710bfa" } modify_external_catalogs: true
W20260320 15:27:19.512164 28041 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 08181eea58a24e1cb424f0e1e10f46ca because the tablet was not found. No further retry: Not found: Tablet not found: 08181eea58a24e1cb424f0e1e10f46ca
I20260320 15:27:19.514616 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1e18f506558543ec8b4ec33dceac311d" } modify_external_catalogs: true
I20260320 15:27:19.514827 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1e18f506558543ec8b4ec33dceac311d" } modify_external_catalogs: true
I20260320 15:27:19.518110 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b85eb77d36ad4cd58e04e60d9db9831c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.518779 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b85eb77d36ad4cd58e04e60d9db9831c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.519170 28729 ts_tablet_manager.cc:1929] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.519327 28729 log.cc:1199] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/cec531361625474f93daa8453b39f944
I20260320 15:27:19.519734 28729 ts_tablet_manager.cc:1950] T cec531361625474f93daa8453b39f944 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.521039 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet cec531361625474f93daa8453b39f944 (table default.Table_c75f89746474457f81d638cfafc1b8d5 [id=715e4038b9a84544ae18e06efcf9edc9]) successfully deleted
I20260320 15:27:19.522084 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_bc859f8772154e8eb9c462cf03069f42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.523265 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc859f8772154e8eb9c462cf03069f42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.529304 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 08181eea58a24e1cb424f0e1e10f46ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.529786 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet cec531361625474f93daa8453b39f944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.530040 28730 tablet_replica.cc:333] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.530256 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:19.530306 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 08181eea58a24e1cb424f0e1e10f46ca because the tablet was not found. No further retry: Not found: Tablet not found: 08181eea58a24e1cb424f0e1e10f46ca
I20260320 15:27:19.530447 28730 raft_consensus.cc:2243] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.530642 28730 raft_consensus.cc:2272] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.531253 28730 ts_tablet_manager.cc:1916] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.535799 28739 raft_consensus.cc:493] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.536005 28739 raft_consensus.cc:515] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.536821 28739 leader_election.cc:290] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.538301 28736 tablet_bootstrap.cc:654] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.538403 28730 ts_tablet_manager.cc:1929] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.538555 28730 log.cc:1199] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/cec531361625474f93daa8453b39f944
I20260320 15:27:19.538489 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.538878 28730 ts_tablet_manager.cc:1950] T cec531361625474f93daa8453b39f944 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.539024 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.539428 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.535748 28751 raft_consensus.cc:493] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.539871 28751 raft_consensus.cc:515] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.540055 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.540571 28751 leader_election.cc:290] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.540807 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet cec531361625474f93daa8453b39f944 (table default.Table_c75f89746474457f81d638cfafc1b8d5 [id=715e4038b9a84544ae18e06efcf9edc9]) successfully deleted
W20260320 15:27:19.541177 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0a7c92a1f4894df586cbb1647ed242a3 because the tablet was not found. No further retry: Not found: Tablet not found: 0a7c92a1f4894df586cbb1647ed242a3
W20260320 15:27:19.541364 28524 leader_election.cc:343] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.541468 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 because the tablet was not found. No further retry: Not found: Tablet not found: 88a005b1c2b54a2fa8515f7a6b165ea8
W20260320 15:27:19.541565 28524 leader_election.cc:343] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.541657 28524 leader_election.cc:304] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:19.541776 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because the tablet was not found. No further retry: Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:19.542908 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e49448a973442ad88b2090527f33899" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.543143 28447 raft_consensus.cc:2468] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:19.544683 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 08181eea58a24e1cb424f0e1e10f46ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.544997 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e49448a973442ad88b2090527f33899" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.545356 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet cec531361625474f93daa8453b39f944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:19.545616 28524 leader_election.cc:343] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.545718 28751 raft_consensus.cc:2749] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.545881 28524 leader_election.cc:304] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:19.546308 28751 raft_consensus.cc:2804] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.546404 28751 raft_consensus.cc:493] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.546509 28751 raft_consensus.cc:3060] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.546900 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.547063 28047 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet f21bb25445644d4686023bf2e6b7e9c5
I20260320 15:27:19.547277 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.547320 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet 47f35263efa3413882954c8c1aa39855 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.547670 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:19.547755 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 87a3d5e4ff8f42409482af49c636be0c has no consensus state
I20260320 15:27:19.548054 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.548370 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.548708 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.549013 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.549342 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.549727 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.549844 28751 raft_consensus.cc:515] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.550081 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.551294 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e49448a973442ad88b2090527f33899" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:19.551869 28524 leader_election.cc:343] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.552672 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef9ab0a1f984e528f0fd7f7ce7a79db" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:19.552947 28739 raft_consensus.cc:493] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.552974 28444 raft_consensus.cc:3060] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.553099 28739 raft_consensus.cc:515] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.553647 28739 leader_election.cc:290] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.553803 28739 raft_consensus.cc:493] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.553910 28739 raft_consensus.cc:515] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.554490 28739 leader_election.cc:290] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.554620 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351a4cc8692a472d9bc5f3dbcc5ee172" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.555361 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "351a4cc8692a472d9bc5f3dbcc5ee172" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:19.556222 28524 leader_election.cc:343] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.556490 28524 leader_election.cc:343] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.556607 28524 leader_election.cc:304] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:19.557619 28735 tablet_replica.cc:333] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:19.557862 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e49448a973442ad88b2090527f33899" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:19.557925 28240 leader_election.cc:343] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: d6ce1fe5280a4e888b8f1bec19ab432f
I20260320 15:27:19.558117 28446 raft_consensus.cc:3060] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.558238 28735 raft_consensus.cc:2243] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:19.558549 28240 leader_election.cc:343] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: d6ce1fe5280a4e888b8f1bec19ab432f
I20260320 15:27:19.558903 28751 leader_election.cc:290] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.559006 28735 pending_rounds.cc:70] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:19.559082 28751 raft_consensus.cc:2749] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.559192 28735 pending_rounds.cc:77] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387720024387584 op_type: NO_OP noop_request { }
I20260320 15:27:19.559412 28735 raft_consensus.cc:2889] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:19.560129 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_57247cf8e07b450984f71c47c56e7b1b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.560777 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57247cf8e07b450984f71c47c56e7b1b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.562549 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3284c73cfa4dfba65a88a7dd243190" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.562911 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_6651aae307a5497b89f3162d8a882f3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.563416 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3284c73cfa4dfba65a88a7dd243190" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:19.563570 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6651aae307a5497b89f3162d8a882f3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:19.564271 28524 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.564574 28524 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.564715 28524 leader_election.cc:304] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:19.565524 28739 raft_consensus.cc:2749] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.566011 28446 raft_consensus.cc:2468] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:19.566723 28041 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 08181eea58a24e1cb424f0e1e10f46ca because the tablet was not found. No further retry: Not found: Tablet not found: 08181eea58a24e1cb424f0e1e10f46ca
I20260320 15:27:19.566881 28524 leader_election.cc:304] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
W20260320 15:27:19.567052 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet cec531361625474f93daa8453b39f944 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cec531361625474f93daa8453b39f944 already in progress: creating tablet
W20260320 15:27:19.567297 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 88a005b1c2b54a2fa8515f7a6b165ea8 already in progress: creating tablet
I20260320 15:27:19.567329 28739 raft_consensus.cc:2804] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:19.567754 28739 raft_consensus.cc:697] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:19.567819 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ef9ab0a1f984e528f0fd7f7ce7a79db" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.567898 28751 raft_consensus.cc:493] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.568027 28751 raft_consensus.cc:515] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.568789 28751 leader_election.cc:290] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.568306 28739 consensus_queue.cc:237] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.569629 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484375918aed45a9bc93e7ba5ae3a113" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.569922 28303 raft_consensus.cc:2468] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:19.570950 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484375918aed45a9bc93e7ba5ae3a113" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.571230 28447 raft_consensus.cc:2468] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:19.571489 28735 raft_consensus.cc:2272] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.571822 28524 leader_election.cc:304] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:19.573089 28444 raft_consensus.cc:2468] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:19.573436 28751 raft_consensus.cc:2804] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.573571 28751 raft_consensus.cc:493] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.573686 28751 raft_consensus.cc:3060] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.574152 28735 ts_tablet_manager.cc:1916] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.576916 28751 raft_consensus.cc:515] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.578368 28736 tablet_bootstrap.cc:492] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:19.578580 28736 ts_tablet_manager.cc:1403] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.101s user 0.000s sys 0.003s
I20260320 15:27:19.579449 28736 raft_consensus.cc:359] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.579711 28736 raft_consensus.cc:385] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.579782 28736 raft_consensus.cc:740] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.580065 28736 consensus_queue.cc:260] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.581157 28734 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.583386 28736 ts_tablet_manager.cc:1434] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:19.584144 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484375918aed45a9bc93e7ba5ae3a113" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:19.584393 28303 raft_consensus.cc:3060] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.585593 28734 ts_tablet_manager.cc:1403] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.153s user 0.013s sys 0.025s
W20260320 15:27:19.587743 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d21b0e8db1a424ea329b8f9cd03ab5b has no consensus state
I20260320 15:27:19.588253 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_36e2b9afc7aa435d91c333f483c1ebc1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.588339 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_cfc52407bf394111b223a209b9adbdec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.588862 28735 ts_tablet_manager.cc:1929] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260320 15:27:19.588919 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36e2b9afc7aa435d91c333f483c1ebc1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.589061 28735 log.cc:1199] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/d6ce1fe5280a4e888b8f1bec19ab432f
W20260320 15:27:19.589586 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfc52407bf394111b223a209b9adbdec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.590209 28734 raft_consensus.cc:359] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.590476 28734 raft_consensus.cc:385] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.590567 28734 raft_consensus.cc:740] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.590967 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7879fae3c7d843cbaebc6ccd995af608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.591943 28734 consensus_queue.cc:260] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.592080 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_856a539607904111b591f60c46495372"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.592832 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_856a539607904111b591f60c46495372 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.596235 28053 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.596504 28053 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_b2423a0d0aa445fb9e50e2c60c6328fa [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 1 got 0
I20260320 15:27:19.597123 28303 raft_consensus.cc:2468] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:19.597608 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7879fae3c7d843cbaebc6ccd995af608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.598203 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet dfaf72b4f25c417cafd2f53db42b3315 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.599977 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a7c92a1f4894df586cbb1647ed242a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.600153 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.600443 28729 tablet_replica.cc:333] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:19.600656 28729 raft_consensus.cc:2243] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.600785 28524 leader_election.cc:304] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:19.600867 28729 raft_consensus.cc:2272] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.599493 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c12e189dde7543a282d5237cf3200a0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.601672 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c12e189dde7543a282d5237cf3200a0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.602532 28729 ts_tablet_manager.cc:1916] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.603798 28751 leader_election.cc:290] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.604065 28751 raft_consensus.cc:2804] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:19.604231 28751 raft_consensus.cc:697] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:19.605036 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "484375918aed45a9bc93e7ba5ae3a113" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:19.605304 28444 raft_consensus.cc:3060] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.605376 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (DEFAULT_TABLE table=default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.604609 28751 consensus_queue.cc:237] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.605849 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.608047 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 87a3d5e4ff8f42409482af49c636be0c (DEFAULT_TABLE table=default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.608590 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a3d5e4ff8f42409482af49c636be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.605389 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.612174 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee4aafbcc4e46bb9dbb0460a8575282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.614483 28444 raft_consensus.cc:2468] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:19.617650 28422 tablet_service.cc:1511] Processing CreateTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 (DEFAULT_TABLE table=default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.618094 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3849a556d94649a3d029b4ac41fcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.621627 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab (DEFAULT_TABLE table=default.Table_3b9ef310f807440f94a4042a41b0ca75 [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.622186 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbd3b0e92a744dcbc132037c67303ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.629936 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.630555 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee4aafbcc4e46bb9dbb0460a8575282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.632138 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (DEFAULT_TABLE table=default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.632651 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.636464 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 87a3d5e4ff8f42409482af49c636be0c (DEFAULT_TABLE table=default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.637034 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a3d5e4ff8f42409482af49c636be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.640249 28563 tablet_service.cc:1511] Processing CreateTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 (DEFAULT_TABLE table=default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.640678 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3849a556d94649a3d029b4ac41fcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.644431 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab (DEFAULT_TABLE table=default.Table_3b9ef310f807440f94a4042a41b0ca75 [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.644892 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbd3b0e92a744dcbc132037c67303ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.652069 28733 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.652451 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet dfaf72b4f25c417cafd2f53db42b3315
I20260320 15:27:19.653021 28053 catalog_manager.cc:6741] ReplaceTablet: tablet dfaf72b4f25c417cafd2f53db42b3315 deleted and replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:19.653296 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.653558 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet df5ffaedeaf94def93d93a538c42e3bd (table default.Table_945b0cecf5834f899f177152b6153d2d [id=aa8e982cd47c499a90f51cad2f96548e]). Expected version 1 got 0
I20260320 15:27:19.653802 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_9c0e5be4f98e48b8aecebbf9db56e138 [id=0466f754fc5f43249331fda298bb726c]). Expected version 1 got 0
I20260320 15:27:19.654017 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (table default.Table_66d9e5b794ae4f1298e4264693af9a3c [id=d9929a996d914fb387ec71330be86b22]). Expected version 1 got 0
I20260320 15:27:19.654207 28051 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_b2423a0d0aa445fb9e50e2c60c6328fa [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 1 got 0
I20260320 15:27:19.654251 28733 tablet_bootstrap.cc:654] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:19.654822 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9dbd3b0e92a744dcbc132037c67303ab consensus state has no leader
I20260320 15:27:19.656524 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 8e3284c73cfa4dfba65a88a7dd243190
I20260320 15:27:19.657431 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:19.661146 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 (DEFAULT_TABLE table=default.Table_1832ad9fbd5449c9bafcf89adc062c0e [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.661798 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ee8b3598384569bbe3d63abb3eaed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.665663 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.666013 28730 tablet_replica.cc:333] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.666244 28730 raft_consensus.cc:2243] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.666527 28730 raft_consensus.cc:2272] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.666916 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.667351 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.667514 28730 ts_tablet_manager.cc:1916] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.667670 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.667786 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet d6ce1fe5280a4e888b8f1bec19ab432f with delete_type TABLET_DATA_DELETED (replaced by tablet dfaf72b4f25c417cafd2f53db42b3315) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.667999 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 with delete_type TABLET_DATA_DELETED (replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.668174 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.668475 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.668565 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.668627 28753 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.668963 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_638e25af5c024f1e93cf9aff051f145f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.669668 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_638e25af5c024f1e93cf9aff051f145f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.670866 28736 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:19.671886 28783 raft_consensus.cc:493] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.672050 28783 raft_consensus.cc:515] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.672832 28783 leader_election.cc:290] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.674495 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b2423a0d0aa445fb9e50e2c60c6328fa" } new_table_name: "default.Table_5e0b71eb6a61483ca92b4f5a12a0682c" modify_external_catalogs: true
I20260320 15:27:19.675643 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f36e145cb8cc41809dbbdb124102844f" } new_table_name: "default.Table_b61482084570428fadb57b2429b92e99" modify_external_catalogs: true
I20260320 15:27:19.678922 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.679028 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.679142 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.679961 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 (DEFAULT_TABLE table=default.Table_1832ad9fbd5449c9bafcf89adc062c0e [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.680055 28565 tablet_service.cc:1511] Processing CreateTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e (DEFAULT_TABLE table=default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.680233 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 (DEFAULT_TABLE table=default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.680446 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ee8b3598384569bbe3d63abb3eaed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.680646 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c939f8c30943fe8f0619b7dcacc7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.680886 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9248f0fee2fd446bb8c53891f6e7ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.669270 28753 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.682647 28753 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.683112 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e (DEFAULT_TABLE table=default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.684377 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9248f0fee2fd446bb8c53891f6e7ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.686292 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 (DEFAULT_TABLE table=default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.687314 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c939f8c30943fe8f0619b7dcacc7e0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:19.687867 28382 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:19.688787 28730 ts_tablet_manager.cc:1929] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.688939 28730 log.cc:1199] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8e3284c73cfa4dfba65a88a7dd243190
W20260320 15:27:19.689035 28038 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0d21b0e8db1a424ea329b8f9cd03ab5b consensus state has no leader
I20260320 15:27:19.690812 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab709738a3f471da773fd9b523f7fbb (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.691013 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab709738a3f471da773fd9b523f7fbb (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.693193 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab709738a3f471da773fd9b523f7fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.696171 28730 ts_tablet_manager.cc:1950] T 8e3284c73cfa4dfba65a88a7dd243190 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.701845 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab709738a3f471da773fd9b523f7fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.702328 28733 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.702512 28730 tablet_replica.cc:333] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.702620 28733 ts_tablet_manager.cc:1403] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.008s
I20260320 15:27:19.705034 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_55de4adf4e2242bf959513e74510cb9d" } modify_external_catalogs: true
I20260320 15:27:19.705281 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_55de4adf4e2242bf959513e74510cb9d" } modify_external_catalogs: true
I20260320 15:27:19.711108 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_856a539607904111b591f60c46495372" } modify_external_catalogs: true
I20260320 15:27:19.711385 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_856a539607904111b591f60c46495372" } modify_external_catalogs: true
I20260320 15:27:19.712983 28730 raft_consensus.cc:2243] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.713269 28730 raft_consensus.cc:2272] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.715806 28730 ts_tablet_manager.cc:1916] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:19.729907 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 484375918aed45a9bc93e7ba5ae3a113 consensus state has no leader
I20260320 15:27:19.731072 28046 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_9c0e5be4f98e48b8aecebbf9db56e138 [id=0466f754fc5f43249331fda298bb726c]). Expected version 1 got 0
W20260320 15:27:19.732967 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 026bb078f3904a44820442e69e654f72 consensus state has no leader
I20260320 15:27:19.735280 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.743346 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 with delete_type TABLET_DATA_DELETED (replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.743391 28564 tablet_service.cc:1511] Processing CreateTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (DEFAULT_TABLE table=default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.744004 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.744011 28567 tablet_service.cc:1511] Processing CreateTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 (DEFAULT_TABLE table=default.Table_36e2b9afc7aa435d91c333f483c1ebc1 [id=f0e9c9ad3d504860a14df78e47af5138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.745328 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce76eeddb583441bb18fa5fb8793ffb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.748164 28729 ts_tablet_manager.cc:1929] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.748422 28729 log.cc:1199] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0a7c92a1f4894df586cbb1647ed242a3
I20260320 15:27:19.749775 28729 ts_tablet_manager.cc:1950] T 0a7c92a1f4894df586cbb1647ed242a3 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.750764 28729 tablet_replica.cc:333] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:19.751591 28733 raft_consensus.cc:359] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.752635 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0a7c92a1f4894df586cbb1647ed242a3 (table default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]) successfully deleted
I20260320 15:27:19.753314 28733 raft_consensus.cc:385] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.754053 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8e3284c73cfa4dfba65a88a7dd243190 (table default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]) successfully deleted
W20260320 15:27:19.754382 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0ab709738a3f471da773fd9b523f7fbb because the tablet was not found. No further retry: Not found: Tablet not found: 0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:19.754985 28558 tablet_service.cc:1511] Processing CreateTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b (DEFAULT_TABLE table=default.Table_bc859f8772154e8eb9c462cf03069f42 [id=51c653187cbb43cdbfccbaba8561a559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.757711 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:19.751297 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 97811e288242436c8e74fd9f45d8d81e (DEFAULT_TABLE table=default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.760046 28729 raft_consensus.cc:2243] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.760241 28729 raft_consensus.cc:2272] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.760300 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97811e288242436c8e74fd9f45d8d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.760883 28734 ts_tablet_manager.cc:1434] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.175s user 0.001s sys 0.001s
I20260320 15:27:19.761833 28729 ts_tablet_manager.cc:1916] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.766975 28734 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.763082 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 2069aa42660947d8a1d37d6d40846991 (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.772125 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4bcf209c9cb44722b4afe02048a4898f" } new_table_name: "default.Table_104cf9fbc3624b2586dfd717214b89a2" modify_external_catalogs: true
I20260320 15:27:19.773916 28734 tablet_bootstrap.cc:654] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.773834 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_307076d582aa44b9be757a3307398132" } modify_external_catalogs: true
I20260320 15:27:19.774055 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_307076d582aa44b9be757a3307398132" } modify_external_catalogs: true
I20260320 15:27:19.774830 28733 raft_consensus.cc:740] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.776885 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:19.777786 28733 consensus_queue.cc:260] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.779129 28417 tablet_service.cc:1558] Processing DeleteTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.779352 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.781787 28729 ts_tablet_manager.cc:1929] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.781960 28729 log.cc:1199] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:19.782330 28729 ts_tablet_manager.cc:1950] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:19.792670 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d474e70ac621425bb25bf63c5cbe3c06" } new_table_name: "default.Table_04f3937bd89341f98bf5a89d8b00de07" modify_external_catalogs: true
I20260320 15:27:19.796555 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.797376 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.800356 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e976aa12e3ab47bd8d42318871768eba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.800962 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e976aa12e3ab47bd8d42318871768eba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.801705 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7da45f21aeef45fbb41fbf00df427cb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.802356 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7da45f21aeef45fbb41fbf00df427cb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.803467 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ee7fbcc0eb0e4f7e961909167e813202"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.804008 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee7fbcc0eb0e4f7e961909167e813202 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.752161 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b (DEFAULT_TABLE table=default.Table_bc859f8772154e8eb9c462cf03069f42 [id=51c653187cbb43cdbfccbaba8561a559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.753014 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc (DEFAULT_TABLE table=default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.755985 28422 tablet_service.cc:1511] Processing CreateTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 (DEFAULT_TABLE table=default.Table_36e2b9afc7aa435d91c333f483c1ebc1 [id=f0e9c9ad3d504860a14df78e47af5138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.757354 28420 tablet_service.cc:1511] Processing CreateTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (DEFAULT_TABLE table=default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.757710 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 2069aa42660947d8a1d37d6d40846991 (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.758433 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.758821 28421 tablet_service.cc:1511] Processing CreateTablet for tablet b395977eb0444bcf8563cd8774fab40f (DEFAULT_TABLE table=default.Table_e5a27bce7ad246d8807536e43ccd8b2e [id=95ea602214894df19afd570e9cf0258d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.759137 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea (DEFAULT_TABLE table=default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.805498 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14be8e6ed89b413a84f9517ca5ff02dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.759471 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 (DEFAULT_TABLE table=default.Table_c12e189dde7543a282d5237cf3200a0c [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.805823 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.808223 28565 tablet_service.cc:1511] Processing CreateTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc (DEFAULT_TABLE table=default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:19.809479 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4346c464fb34e9dae8aeccc99dc07a3 consensus state has no leader
I20260320 15:27:19.810024 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:19.811812 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1268b25a1c99404aa77e5b1bcc415f48. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:19.817147 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 801b3b0ba2344cbe8d89111e17e546ee consensus state has no leader
I20260320 15:27:19.819856 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.823377 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75775842ffbb4ff1a61ba25bf6906c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.828140 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd (DEFAULT_TABLE table=default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.828487 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038fd8c34ee48278404acfec3e0b8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.828193 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.829021 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825f7e3fdc004fe1a3bcca6deb20cccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.831713 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce76eeddb583441bb18fa5fb8793ffb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.831997 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea (DEFAULT_TABLE table=default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.835951 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 97811e288242436c8e74fd9f45d8d81e (DEFAULT_TABLE table=default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.836531 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.836797 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.837074 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993c9b6ae14ae98c2239abd65333ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.839702 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395977eb0444bcf8563cd8774fab40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.847013 28733 ts_tablet_manager.cc:1434] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.076s user 0.002s sys 0.001s
I20260320 15:27:19.847510 28733 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.847682 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2069aa42660947d8a1d37d6d40846991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.849050 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97811e288242436c8e74fd9f45d8d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.850435 28733 tablet_bootstrap.cc:654] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.851454 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2069aa42660947d8a1d37d6d40846991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.852178 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75775842ffbb4ff1a61ba25bf6906c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.852653 28567 tablet_service.cc:1511] Processing CreateTablet for tablet b395977eb0444bcf8563cd8774fab40f (DEFAULT_TABLE table=default.Table_e5a27bce7ad246d8807536e43ccd8b2e [id=95ea602214894df19afd570e9cf0258d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.853106 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395977eb0444bcf8563cd8774fab40f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:19.855129 28239 leader_election.cc:343] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 0a7c92a1f4894df586cbb1647ed242a3
I20260320 15:27:19.855578 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b7f757ebdf456eb81085e5be54da4b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.857035 28735 ts_tablet_manager.cc:1950] T d6ce1fe5280a4e888b8f1bec19ab432f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:19.857318 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_dabfce1f58cf47eeb405fae7b61d20c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.857427 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3187a7e0ff66453abc8f3bba9c6bad68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.858037 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3187a7e0ff66453abc8f3bba9c6bad68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:19.858042 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dabfce1f58cf47eeb405fae7b61d20c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.861207 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_501982d031484c3eb2382cf055a9d3be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.861785 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_501982d031484c3eb2382cf055a9d3be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.862439 28735 tablet_replica.cc:333] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:19.862623 28735 raft_consensus.cc:2243] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.862938 28735 raft_consensus.cc:2272] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.863094 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_97a6eb082b53476a8140573e2c89dfca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:19.863651 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97a6eb082b53476a8140573e2c89dfca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.864892 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2f4e5acc5d2048e6abbd592ff5d537e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.865268 28736 tablet_bootstrap.cc:654] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:19.865446 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f4e5acc5d2048e6abbd592ff5d537e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.867251 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.866756 28049 catalog_manager.cc:5671] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:19.867506 28735 ts_tablet_manager.cc:1916] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.867507 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd (DEFAULT_TABLE table=default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.868031 28049 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]). Expected version 2 got 0
I20260320 15:27:19.868372 28049 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_5e0b71eb6a61483ca92b4f5a12a0682c [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 2 got 0
I20260320 15:27:19.868389 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14be8e6ed89b413a84f9517ca5ff02dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.870978 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825f7e3fdc004fe1a3bcca6deb20cccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.881245 28798 raft_consensus.cc:493] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.881449 28798 raft_consensus.cc:515] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.882210 28798 leader_election.cc:290] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.884527 28730 ts_tablet_manager.cc:1929] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.884815 28730 log.cc:1199] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/4902ffdcdbc54d5aa9c71f6cd239c69a
I20260320 15:27:19.889531 28735 ts_tablet_manager.cc:1929] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.890098 28730 ts_tablet_manager.cc:1950] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.890650 28739 raft_consensus.cc:493] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.890794 28730 tablet_replica.cc:333] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:19.889189 28733 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.891080 28733 ts_tablet_manager.cc:1403] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.005s
I20260320 15:27:19.891036 28739 raft_consensus.cc:515] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.891978 28739 leader_election.cc:290] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.892346 28733 raft_consensus.cc:359] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.892642 28733 raft_consensus.cc:385] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.892741 28733 raft_consensus.cc:740] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.893081 28733 consensus_queue.cc:260] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.894181 28733 ts_tablet_manager.cc:1434] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:19.894615 28733 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:19.887974 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993c9b6ae14ae98c2239abd65333ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.895423 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 81a1e0957439458083e3a86ac2f01b1e (DEFAULT_TABLE table=default.Table_6651aae307a5497b89f3162d8a882f3f [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.895910 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a1e0957439458083e3a86ac2f01b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.897110 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038fd8c34ee48278404acfec3e0b8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.886945 28791 raft_consensus.cc:493] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.897981 28791 raft_consensus.cc:515] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.898939 28791 leader_election.cc:290] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.869362 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6651aae307a5497b89f3162d8a882f3f" } new_table_name: "default.Table_30bff6fd60bc40589933b67acaa11868" modify_external_catalogs: true
I20260320 15:27:19.873474 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ebc7ffacd75e49a4b2ed94aafff57936" } modify_external_catalogs: true
I20260320 15:27:19.900799 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ebc7ffacd75e49a4b2ed94aafff57936" } modify_external_catalogs: true
I20260320 15:27:19.873870 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_066f88ea62914d6f93053e540c6eae61" } modify_external_catalogs: true
I20260320 15:27:19.901991 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_066f88ea62914d6f93053e540c6eae61" } modify_external_catalogs: true
I20260320 15:27:19.894320 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (table default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]) successfully deleted
W20260320 15:27:19.909652 28383 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.909816 28383 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:19.910946 28383 raft_consensus.cc:2686] T 22b7f757ebdf456eb81085e5be54da4b P d22e73471b9046d28d3cf63c905b11b7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:19.912425 28753 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.912585 28753 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.914433 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 (DEFAULT_TABLE table=default.Table_c12e189dde7543a282d5237cf3200a0c [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.914515 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 81a1e0957439458083e3a86ac2f01b1e (DEFAULT_TABLE table=default.Table_6651aae307a5497b89f3162d8a882f3f [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.914383 28795 raft_consensus.cc:493] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.915120 28795 raft_consensus.cc:515] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:19.915737 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 54809.7us
I20260320 15:27:19.916003 28795 leader_election.cc:290] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.916283 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2586d40343b4365ab8a67cce2e1c6ca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:19.916458 28730 raft_consensus.cc:2243] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.916502 28587 raft_consensus.cc:2468] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:19.916594 28753 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.917428 28382 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:19.917775 28753 raft_consensus.cc:2804] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.916692 28730 raft_consensus.cc:2272] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.918550 28753 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.918569 28730 ts_tablet_manager.cc:1916] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.918661 28753 raft_consensus.cc:3060] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.919111 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a1e0957439458083e3a86ac2f01b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.922477 28753 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:19.923609 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81a1e0957439458083e3a86ac2f01b1e consensus state has no leader
I20260320 15:27:19.925485 28730 ts_tablet_manager.cc:1929] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:19.925652 28730 log.cc:1199] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/d6ce1fe5280a4e888b8f1bec19ab432f
I20260320 15:27:19.926064 28730 ts_tablet_manager.cc:1950] T d6ce1fe5280a4e888b8f1bec19ab432f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:19.926976 28733 tablet_bootstrap.cc:654] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.927423 28753 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:19.937002 28733 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:19.937178 28733 ts_tablet_manager.cc:1403] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.001s
I20260320 15:27:19.938350 28733 raft_consensus.cc:359] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.938567 28733 raft_consensus.cc:385] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.938656 28733 raft_consensus.cc:740] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.938903 28733 consensus_queue.cc:260] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.939551 28733 ts_tablet_manager.cc:1434] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:19.940656 28795 raft_consensus.cc:493] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.940821 28795 raft_consensus.cc:515] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.941190 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0b295d872d094ac3bca1ea9effaf3081"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:19.941439 28795 leader_election.cc:290] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.941686 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 47f35263efa3413882954c8c1aa39855 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:19.941841 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b295d872d094ac3bca1ea9effaf3081 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:19.942257 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_945b0cecf5834f899f177152b6153d2d" } new_table_name: "default.Table_d7cfd4b57bfc44ef8c522d075251b762" modify_external_catalogs: true
I20260320 15:27:19.942593 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 484375918aed45a9bc93e7ba5ae3a113 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.943004 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 4902ffdcdbc54d5aa9c71f6cd239c69a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.943591 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5e0b71eb6a61483ca92b4f5a12a0682c" } new_table_name: "default.Table_a5b9cd3e79294e4481b46a6421326ac1" modify_external_catalogs: true
I20260320 15:27:19.944717 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet 351a4cc8692a472d9bc5f3dbcc5ee172 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.944794 28795 raft_consensus.cc:493] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:19.944936 28795 raft_consensus.cc:515] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.945688 28795 leader_election.cc:290] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:19.947574 28524 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:19.948448 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet 0ee4aafbcc4e46bb9dbb0460a8575282 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:19.948969 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:19.948983 28729 tablet_replica.cc:333] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:19.949177 28729 raft_consensus.cc:2243] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:19.949359 28729 raft_consensus.cc:2272] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:19.950366 28734 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:19.950532 28734 ts_tablet_manager.cc:1403] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.184s user 0.011s sys 0.007s
I20260320 15:27:19.951287 28734 raft_consensus.cc:359] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.951473 28734 raft_consensus.cc:385] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:19.951606 28734 raft_consensus.cc:740] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:19.951838 28734 consensus_queue.cc:260] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.952333 28734 ts_tablet_manager.cc:1434] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:19.952647 28734 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:19.954787 28734 tablet_bootstrap.cc:654] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:19.958789 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "801b3b0ba2344cbe8d89111e17e546ee" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.959350 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5ffaedeaf94def93d93a538c42e3bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.959918 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7e9568395c44af2912efb175fc1704a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.960368 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbd3b0e92a744dcbc132037c67303ab" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.960882 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b08bcbee8541c998a23dbc1947796a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:19.961087 28444 raft_consensus.cc:2468] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:19.961481 28524 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
W20260320 15:27:19.961879 28524 leader_election.cc:343] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.962198 28524 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.962528 28524 leader_election.cc:343] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:19.962824 28524 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:19.962946 28524 leader_election.cc:304] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:19.963393 28739 raft_consensus.cc:2749] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:19.963866 28524 leader_election.cc:304] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:19.964145 28739 raft_consensus.cc:2804] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:19.964258 28739 raft_consensus.cc:493] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:19.964350 28739 raft_consensus.cc:3060] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:19.967150 28739 raft_consensus.cc:515] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.968088 28739 leader_election.cc:290] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:19.970525 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 22b7f757ebdf456eb81085e5be54da4b (table default.Table_9c0e5be4f98e48b8aecebbf9db56e138 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
W20260320 15:27:19.970817 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 8e3284c73cfa4dfba65a88a7dd243190 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e3284c73cfa4dfba65a88a7dd243190 already in progress: creating tablet
I20260320 15:27:19.970880 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_adca08e7f2314aec8ad43dc91db184de" } modify_external_catalogs: true
W20260320 15:27:19.971146 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0ab709738a3f471da773fd9b523f7fbb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ab709738a3f471da773fd9b523f7fbb already in progress: creating tablet
I20260320 15:27:19.971370 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_adca08e7f2314aec8ad43dc91db184de" } modify_external_catalogs: true
I20260320 15:27:19.973500 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b08bcbee8541c998a23dbc1947796a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:19.973745 28444 raft_consensus.cc:3060] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:19.975199 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df5ffaedeaf94def93d93a538c42e3bd consensus state has no leader
I20260320 15:27:19.976753 28444 raft_consensus.cc:2468] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:19.977666 28524 leader_election.cc:304] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:19.978183 28739 raft_consensus.cc:2804] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:19.978106 28050 catalog_manager.cc:5671] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:19.978444 28739 raft_consensus.cc:697] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:19.978600 28050 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_5e0b71eb6a61483ca92b4f5a12a0682c [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 2 got 0
I20260320 15:27:19.979020 28739 consensus_queue.cc:237] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:19.979717 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 351a4cc8692a472d9bc5f3dbcc5ee172
I20260320 15:27:19.980146 28047 catalog_manager.cc:6741] ReplaceTablet: tablet 351a4cc8692a472d9bc5f3dbcc5ee172 deleted and replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0
I20260320 15:27:19.980566 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.980804 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 8e49448a973442ad88b2090527f33899
I20260320 15:27:19.981088 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:19.981755 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 484375918aed45a9bc93e7ba5ae3a113 consensus state has no leader
I20260320 15:27:19.983057 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e49448a973442ad88b2090527f33899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.983296 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 with delete_type TABLET_DATA_DELETED (replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:19.988444 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.989027 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 with delete_type TABLET_DATA_DELETED (replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.890096 28283 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.989507 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e49448a973442ad88b2090527f33899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:19.989653 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee4aafbcc4e46bb9dbb0460a8575282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.990963 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3187a7e0ff66453abc8f3bba9c6bad68" } modify_external_catalogs: true
I20260320 15:27:19.991180 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3187a7e0ff66453abc8f3bba9c6bad68" } modify_external_catalogs: true
I20260320 15:27:19.891148 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (DEFAULT_TABLE table=default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.991942 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.992883 28729 ts_tablet_manager.cc:1916] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:19.892024 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 87a3d5e4ff8f42409482af49c636be0c (DEFAULT_TABLE table=default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.893651 28279 tablet_service.cc:1511] Processing CreateTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 (DEFAULT_TABLE table=default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.894280 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab (DEFAULT_TABLE table=default.Table_3b9ef310f807440f94a4042a41b0ca75 [id=634e24fdc81640fbb393a4b19bac7343]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.995885 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87a3d5e4ff8f42409482af49c636be0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.996161 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dbd3b0e92a744dcbc132037c67303ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.894735 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 (DEFAULT_TABLE table=default.Table_1832ad9fbd5449c9bafcf89adc062c0e [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.996552 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51ee8b3598384569bbe3d63abb3eaed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:19.899329 28275 tablet_service.cc:1558] Processing DeleteTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 with delete_type TABLET_DATA_DELETED (replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.902866 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.904670 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:19.906041 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 0ab709738a3f471da773fd9b523f7fbb (DEFAULT_TABLE table=default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.906036 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e (DEFAULT_TABLE table=default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.907356 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 (DEFAULT_TABLE table=default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.917127 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2586d40343b4365ab8a67cce2e1c6ca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.926478 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2586d40343b4365ab8a67cce2e1c6ca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:19.940735 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5ffaedeaf94def93d93a538c42e3bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.999760 28271 tablet_service.cc:1511] Processing CreateTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (DEFAULT_TABLE table=default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.000451 28284 tablet_service.cc:1511] Processing CreateTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 (DEFAULT_TABLE table=default.Table_36e2b9afc7aa435d91c333f483c1ebc1 [id=f0e9c9ad3d504860a14df78e47af5138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.000558 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc (DEFAULT_TABLE table=default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.001075 28269 tablet_service.cc:1511] Processing CreateTablet for tablet b395977eb0444bcf8563cd8774fab40f (DEFAULT_TABLE table=default.Table_e5a27bce7ad246d8807536e43ccd8b2e [id=95ea602214894df19afd570e9cf0258d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.001195 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea (DEFAULT_TABLE table=default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.001696 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 (DEFAULT_TABLE table=default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.941200 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa22d207ad8a4555aa9573b15fa1fe1a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.941529 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "801b3b0ba2344cbe8d89111e17e546ee" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.941802 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7e9568395c44af2912efb175fc1704a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.002671 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b (DEFAULT_TABLE table=default.Table_bc859f8772154e8eb9c462cf03069f42 [id=51c653187cbb43cdbfccbaba8561a559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:19.947906 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b08bcbee8541c998a23dbc1947796a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:19.969012 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2b08bcbee8541c998a23dbc1947796a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:19.981732 28736 tablet_bootstrap.cc:492] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.003180 28736 ts_tablet_manager.cc:1403] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.403s user 0.006s sys 0.003s
I20260320 15:27:19.988817 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 2069aa42660947d8a1d37d6d40846991 (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.004249 28524 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.004439 28524 leader_election.cc:304] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:20.005754 28524 leader_election.cc:343] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.005913 28524 leader_election.cc:304] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.006304 28798 raft_consensus.cc:2749] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.006515 28798 raft_consensus.cc:2749] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:20.006953 28383 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:20.008107 28524 leader_election.cc:343] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.008257 28524 leader_election.cc:304] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.008671 28798 raft_consensus.cc:2749] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:20.009332 28524 leader_election.cc:343] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:20.010030 28524 leader_election.cc:343] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.011647 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea3849a556d94649a3d029b4ac41fcf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.012737 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2586d40343b4365ab8a67cce2e1c6ca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:20.012804 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bc859f8772154e8eb9c462cf03069f42" } new_table_name: "default.Table_a30a799b46f848b98910771717144df3" modify_external_catalogs: true
I20260320 15:27:20.012979 28587 raft_consensus.cc:3060] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:20.014876 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1268b25a1c99404aa77e5b1bcc415f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.016388 28587 raft_consensus.cc:2468] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:20.017192 28049 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 9ebc1f51daaa484987a6d380dd823cd3
I20260320 15:27:20.018370 28382 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
W20260320 15:27:20.019061 28383 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.020596 28730 tablet_replica.cc:333] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.022156 28730 raft_consensus.cc:2243] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.022424 28730 raft_consensus.cc:2272] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:20.022958 28524 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.023445 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_66d9e5b794ae4f1298e4264693af9a3c" } modify_external_catalogs: true
I20260320 15:27:20.023697 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_66d9e5b794ae4f1298e4264693af9a3c" } modify_external_catalogs: true
I20260320 15:27:20.025045 28733 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.025820 28730 ts_tablet_manager.cc:1916] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.026075 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ab709738a3f471da773fd9b523f7fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.026615 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_47ac32f0123e4e8e87fff99efa39eca0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:20.027029 28733 tablet_bootstrap.cc:654] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.027081 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9248f0fee2fd446bb8c53891f6e7ba6e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:20.027287 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47ac32f0123e4e8e87fff99efa39eca0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.027776 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d7cfd4b57bfc44ef8c522d075251b762" } new_table_name: "default.Table_a719aa4da12a4d159826d615a6a2302a" modify_external_catalogs: true
I20260320 15:27:20.025049 28524 leader_election.cc:304] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.028923 28739 raft_consensus.cc:2749] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.032855 28736 raft_consensus.cc:359] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.038479 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c939f8c30943fe8f0619b7dcacc7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.038748 28736 raft_consensus.cc:385] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.043344 28736 raft_consensus.cc:740] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.043694 28736 consensus_queue.cc:260] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.045696 28736 ts_tablet_manager.cc:1434] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.041s user 0.002s sys 0.000s
I20260320 15:27:20.046224 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3038fd8c34ee48278404acfec3e0b8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.052275 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce76eeddb583441bb18fa5fb8793ffb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.056841 28565 tablet_service.cc:1511] Processing CreateTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e (DEFAULT_TABLE table=default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.056962 28559 tablet_service.cc:1511] Processing CreateTablet for tablet da957aed322f46acaf5876c567c2ff94 (DEFAULT_TABLE table=default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.057412 28558 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c (DEFAULT_TABLE table=default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.057581 28739 raft_consensus.cc:493] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.057590 28568 tablet_service.cc:1511] Processing CreateTablet for tablet f640a6ae307140d2aee3992333e96d84 (DEFAULT_TABLE table=default.Table_97a6eb082b53476a8140573e2c89dfca [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.057806 28739 raft_consensus.cc:515] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.058590 28739 leader_election.cc:290] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.057647 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da957aed322f46acaf5876c567c2ff94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.059201 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640a6ae307140d2aee3992333e96d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.059954 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_764f1d7877d24d50b17c6d27527fdd6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.060599 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_764f1d7877d24d50b17c6d27527fdd6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.061692 28753 raft_consensus.cc:2804] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:20.061805 28734 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:20.062083 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6165eab6b0442fa9d94ce577c1b43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.062063 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d135759507f0411dbdc9e8a3608562d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.062603 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d135759507f0411dbdc9e8a3608562d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.062721 28734 ts_tablet_manager.cc:1403] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.109s user 0.025s sys 0.023s
I20260320 15:27:20.063716 28734 raft_consensus.cc:359] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.063974 28734 raft_consensus.cc:385] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.064066 28734 raft_consensus.cc:740] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.064392 28734 consensus_queue.cc:260] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.065053 28269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b395977eb0444bcf8563cd8774fab40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.068022 28736 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.068080 28729 ts_tablet_manager.cc:1929] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.068259 28729 log.cc:1199] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/e7e9568395c44af2912efb175fc1704a
I20260320 15:27:20.068487 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75775842ffbb4ff1a61ba25bf6906c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.070009 28729 ts_tablet_manager.cc:1950] T e7e9568395c44af2912efb175fc1704a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.072045 28729 tablet_replica.cc:333] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:20.072455 28729 raft_consensus.cc:2243] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.075740 28729 raft_consensus.cc:2272] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:20.078068 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df5ffaedeaf94def93d93a538c42e3bd consensus state has no leader
I20260320 15:27:20.078542 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_50e316065f1e43459edfe03e6c2468b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.079245 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50e316065f1e43459edfe03e6c2468b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.081012 28729 ts_tablet_manager.cc:1916] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:20.081168 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75775842ffbb4ff1a61ba25bf6906c4b consensus state has no leader
I20260320 15:27:20.084954 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545c8fd2d4c845adba6dbf42dfddba3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.085619 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 2548d7b8d31049519a85490772621352 (DEFAULT_TABLE table=default.Table_2f4e5acc5d2048e6abbd592ff5d537e4 [id=ba466e4f2691499c801ab1e748f61aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.086889 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993c9b6ae14ae98c2239abd65333ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.089057 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2069aa42660947d8a1d37d6d40846991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.091169 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2548d7b8d31049519a85490772621352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.093504 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.095610 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14be8e6ed89b413a84f9517ca5ff02dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.096944 28735 log.cc:1199] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0a7c92a1f4894df586cbb1647ed242a3
I20260320 15:27:20.099782 28561 tablet_service.cc:1511] Processing CreateTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.100518 28053 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 03d9ae931a354bc2b8e2eb190ca0cff0
I20260320 15:27:20.106105 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d4066b167fcb4656b497ac2ef62d5366"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.106850 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4066b167fcb4656b497ac2ef62d5366 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.109800 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet e7e9568395c44af2912efb175fc1704a (table default.Table_0c2bc058015240478a72fe93a91a9727 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
W20260320 15:27:20.110128 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9248f0fee2fd446bb8c53891f6e7ba6e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9248f0fee2fd446bb8c53891f6e7ba6e already in progress: creating tablet
W20260320 15:27:20.110399 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd already in progress: creating tablet
W20260320 15:27:20.110605 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9b993c9b6ae14ae98c2239abd65333ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b993c9b6ae14ae98c2239abd65333ea already in progress: creating tablet
I20260320 15:27:20.111030 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_714d170555c642ddaa667c974b68cbb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.111809 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_714d170555c642ddaa667c974b68cbb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.113667 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e8f37a490a4fdfbc2a3dc56f6da297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.113705 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_69c8a46cb085490e8a0e7b689e10fa36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.114411 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69c8a46cb085490e8a0e7b689e10fa36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.115971 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3849a556d94649a3d029b4ac41fcf4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.117129 28730 ts_tablet_manager.cc:1929] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.119400 28730 log.cc:1199] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/47f35263efa3413882954c8c1aa39855
I20260320 15:27:20.124240 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 484375918aed45a9bc93e7ba5ae3a113
I20260320 15:27:20.124840 28052 catalog_manager.cc:6741] ReplaceTablet: tablet 484375918aed45a9bc93e7ba5ae3a113 deleted and replaced by tablet 8f6f683638874e90b0d3c8969c702a5c
I20260320 15:27:20.125348 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a219d3248ce44f78a7989f8933d4d3fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:20.125881 28730 ts_tablet_manager.cc:1950] T 47f35263efa3413882954c8c1aa39855 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:20.126092 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a219d3248ce44f78a7989f8933d4d3fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.126937 28730 tablet_replica.cc:333] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.127147 28730 raft_consensus.cc:2243] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:20.127779 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0ee4aafbcc4e46bb9dbb0460a8575282
I20260320 15:27:20.128242 28046 catalog_manager.cc:6741] ReplaceTablet: tablet 0ee4aafbcc4e46bb9dbb0460a8575282 deleted and replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06
I20260320 15:27:20.128584 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_9c0b3e51cc4042fab00ee1662295787e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.129160 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c0b3e51cc4042fab00ee1662295787e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.130849 28048 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_0c2bc058015240478a72fe93a91a9727 [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 4 got 0
W20260320 15:27:20.132493 28524 leader_election.cc:343] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.134153 28730 pending_rounds.cc:70] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:20.134369 28730 pending_rounds.cc:77] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387722156802048 op_type: NO_OP noop_request { }
I20260320 15:27:20.134511 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 47f35263efa3413882954c8c1aa39855 (table default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]) successfully deleted
I20260320 15:27:20.134593 28730 raft_consensus.cc:2889] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:20.134716 28730 raft_consensus.cc:2272] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.135417 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 with delete_type TABLET_DATA_DELETED (replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.137713 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 (DEFAULT_TABLE table=default.Table_c12e189dde7543a282d5237cf3200a0c [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.139724 28730 ts_tablet_manager.cc:1916] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.142935 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 19c939f8c30943fe8f0619b7dcacc7e0
I20260320 15:27:20.143821 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2f4e5acc5d2048e6abbd592ff5d537e4" } new_table_name: "default.Table_fc3c9bc540ae45f1a060eb8ac0b82832" modify_external_catalogs: true
I20260320 15:27:20.146317 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.149017 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f6f683638874e90b0d3c8969c702a5c) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.149405 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 97811e288242436c8e74fd9f45d8d81e (DEFAULT_TABLE table=default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.149920 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97811e288242436c8e74fd9f45d8d81e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.154434 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.154961 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d1db9963c4645ae9aab72b4b22d6938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.162042 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet fa22d207ad8a4555aa9573b15fa1fe1a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.163808 28735 ts_tablet_manager.cc:1950] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.164175 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1268b25a1c99404aa77e5b1bcc415f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.165606 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 9248f0fee2fd446bb8c53891f6e7ba6e from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.168958 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.170085 28733 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.170363 28733 ts_tablet_manager.cc:1403] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.147s user 0.019s sys 0.022s
I20260320 15:27:20.170461 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 203c619513a146f49a4825962f169f45 (DEFAULT_TABLE table=default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.171689 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.171948 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_04f3937bd89341f98bf5a89d8b00de07" } modify_external_catalogs: true
I20260320 15:27:20.172180 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_04f3937bd89341f98bf5a89d8b00de07" } modify_external_catalogs: true
I20260320 15:27:20.173280 28733 raft_consensus.cc:359] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.173590 28733 raft_consensus.cc:385] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.174408 28733 raft_consensus.cc:740] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.175740 28733 consensus_queue.cc:260] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.175923 28753 raft_consensus.cc:697] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:20.178061 28733 ts_tablet_manager.cc:1434] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:20.179064 28046 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:20.176301 28420 tablet_service.cc:1511] Processing CreateTablet for tablet f640a6ae307140d2aee3992333e96d84 (DEFAULT_TABLE table=default.Table_97a6eb082b53476a8140573e2c89dfca [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.179306 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e (DEFAULT_TABLE table=default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.180400 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203c619513a146f49a4825962f169f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.182651 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd (DEFAULT_TABLE table=default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.186658 28733 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.188360 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b (DEFAULT_TABLE table=default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.188830 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_549f77fe41f947ab8e14fbce182049f8" } modify_external_catalogs: true
I20260320 15:27:20.189033 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_549f77fe41f947ab8e14fbce182049f8" } modify_external_catalogs: true
I20260320 15:27:20.192643 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.192972 28300 raft_consensus.cc:1275] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Refusing update from remote peer 80636bb608a1431b955576e296ff63bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:20.193882 28422 tablet_service.cc:1511] Processing CreateTablet for tablet da957aed322f46acaf5876c567c2ff94 (DEFAULT_TABLE table=default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.194365 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2548d7b8d31049519a85490772621352 consensus state has no leader
I20260320 15:27:20.195305 28733 tablet_bootstrap.cc:654] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.195617 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 801b3b0ba2344cbe8d89111e17e546ee
I20260320 15:27:20.196986 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 825f7e3fdc004fe1a3bcca6deb20cccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.198872 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.199690 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 with delete_type TABLET_DATA_DELETED (replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.199980 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e49448a973442ad88b2090527f33899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.200990 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a005b1c2b54a2fa8515f7a6b165ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet e7e9568395c44af2912efb175fc1704a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.201442 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e (DEFAULT_TABLE table=default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.201493 28739 consensus_queue.cc:1048] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:20.201977 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545c8fd2d4c845adba6dbf42dfddba3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.207700 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_36e2b9afc7aa435d91c333f483c1ebc1" } new_table_name: "default.Table_f325d0702e60408fb5a7d3a21290c416" modify_external_catalogs: true
I20260320 15:27:20.208375 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db
I20260320 15:27:20.208405 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_30bff6fd60bc40589933b67acaa11868" } new_table_name: "default.Table_0a6c79ec44334f829ff10cfe247a257e" modify_external_catalogs: true
I20260320 15:27:20.208961 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_dfa57bec1c394719acd3a8098ed1905d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.210738 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfa57bec1c394719acd3a8098ed1905d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.212772 28273 tablet_service.cc:1511] Processing CreateTablet for tablet f640a6ae307140d2aee3992333e96d84 (DEFAULT_TABLE table=default.Table_97a6eb082b53476a8140573e2c89dfca [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.212967 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet e2586d40343b4365ab8a67cce2e1c6ca from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.217345 28734 ts_tablet_manager.cc:1434] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.154s user 0.010s sys 0.022s
W20260320 15:27:20.218816 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81a1e0957439458083e3a86ac2f01b1e consensus state has no leader
W20260320 15:27:20.219054 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce76eeddb583441bb18fa5fb8793ffb0 consensus state has no leader
I20260320 15:27:20.219990 28418 tablet_service.cc:1511] Processing CreateTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.220270 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c (DEFAULT_TABLE table=default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.226202 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f6f683638874e90b0d3c8969c702a5c) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.226322 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b (DEFAULT_TABLE table=default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.228174 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 with delete_type TABLET_DATA_DELETED (replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.228742 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 2548d7b8d31049519a85490772621352 (DEFAULT_TABLE table=default.Table_2f4e5acc5d2048e6abbd592ff5d537e4 [id=ba466e4f2691499c801ab1e748f61aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.230307 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.229897 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 203c619513a146f49a4825962f169f45 (DEFAULT_TABLE table=default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.232211 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.233618 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.233829 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.234071 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_714d170555c642ddaa667c974b68cbb7" } modify_external_catalogs: true
I20260320 15:27:20.234262 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_714d170555c642ddaa667c974b68cbb7" } modify_external_catalogs: true
I20260320 15:27:20.234503 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.235581 28730 ts_tablet_manager.cc:1929] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:20.237488 28729 ts_tablet_manager.cc:1929] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.237728 28729 log.cc:1199] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/e2b08bcbee8541c998a23dbc1947796a
I20260320 15:27:20.238132 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.238346 28783 raft_consensus.cc:493] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.238673 28730 log.cc:1199] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/e2b08bcbee8541c998a23dbc1947796a
I20260320 15:27:20.238976 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c1dcc311a14b33a289ae2b8a485d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.240300 28729 ts_tablet_manager.cc:1950] T e2b08bcbee8541c998a23dbc1947796a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.240792 28753 consensus_queue.cc:237] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.242434 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640a6ae307140d2aee3992333e96d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.247079 28730 ts_tablet_manager.cc:1950] T e2b08bcbee8541c998a23dbc1947796a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.248740 28739 raft_consensus.cc:493] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.248935 28739 raft_consensus.cc:515] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.251665 28052 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:20.259073 28735 tablet_replica.cc:333] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:20.260526 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e822249154a1784ebd66f8e56abdf (DEFAULT_TABLE table=default.Table_a219d3248ce44f78a7989f8933d4d3fa [id=90afb14544844334aff0e82fa156d6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.261063 28413 tablet_service.cc:1511] Processing CreateTablet for tablet da860b779a174adbb4fd6f7f7520348c (DEFAULT_TABLE table=default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.261494 28416 tablet_service.cc:1511] Processing CreateTablet for tablet d57566e7907c4783aafce347e46b89e3 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.265328 28744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260320 15:27:20.268885 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc
I20260320 15:27:20.270691 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea3849a556d94649a3d029b4ac41fcf4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:20.271507 28447 raft_consensus.cc:1275] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Refusing update from remote peer 80636bb608a1431b955576e296ff63bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:20.271993 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfaf72b4f25c417cafd2f53db42b3315" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:20.272320 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.273075 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640a6ae307140d2aee3992333e96d84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:20.273962 28744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:20.274107 28744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260320 15:27:20.276050 28734 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:20.276602 28729 tablet_replica.cc:333] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:20.276805 28729 raft_consensus.cc:2243] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.276997 28729 raft_consensus.cc:2272] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.277637 28729 ts_tablet_manager.cc:1916] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.278240 28734 tablet_bootstrap.cc:654] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.282707 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da957aed322f46acaf5876c567c2ff94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.283763 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet e2b08bcbee8541c998a23dbc1947796a (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
W20260320 15:27:20.284133 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 3038fd8c34ee48278404acfec3e0b8d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3038fd8c34ee48278404acfec3e0b8d8 already in progress: creating tablet
I20260320 15:27:20.284929 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_fc3c9bc540ae45f1a060eb8ac0b82832" } new_table_name: "default.Table_a202f09653ee4487a0f5056420b9a8e1" modify_external_catalogs: true
I20260320 15:27:20.287911 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e8f37a490a4fdfbc2a3dc56f6da297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.291582 28739 leader_election.cc:290] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:20.291633 28524 leader_election.cc:343] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.291707 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c1dcc311a14b33a289ae2b8a485d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.291772 28524 leader_election.cc:304] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.292212 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6165eab6b0442fa9d94ce577c1b43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.292455 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d1db9963c4645ae9aab72b4b22d6938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.292877 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203c619513a146f49a4825962f169f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.295765 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2548d7b8d31049519a85490772621352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.296270 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d57566e7907c4783aafce347e46b89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.300020 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e822249154a1784ebd66f8e56abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.304764 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da860b779a174adbb4fd6f7f7520348c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:20.307065 28744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260320 15:27:20.307171 28744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:20.307251 28744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260320 15:27:20.307315 28744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260320 15:27:20.308818 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e5a27bce7ad246d8807536e43ccd8b2e" } new_table_name: "default.Table_bce978bcf8414d928d5f77eaf4e90cbe" modify_external_catalogs: true
W20260320 15:27:20.310359 28524 leader_election.cc:343] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.310564 28798 raft_consensus.cc:2749] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.310940 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 81a1e0957439458083e3a86ac2f01b1e (DEFAULT_TABLE table=default.Table_6651aae307a5497b89f3162d8a882f3f [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.311179 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7c20a59bd0eb41739dfa0770f3cff465"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:20.311414 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a1e0957439458083e3a86ac2f01b1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.311827 28736 tablet_bootstrap.cc:654] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:20.311827 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c20a59bd0eb41739dfa0770f3cff465 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.312288 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c" } modify_external_catalogs: true
I20260320 15:27:20.312481 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c" } modify_external_catalogs: true
I20260320 15:27:20.313060 28783 raft_consensus.cc:515] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.313779 28783 leader_election.cc:290] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:20.314375 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfaf72b4f25c417cafd2f53db42b3315" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.314507 28730 tablet_replica.cc:333] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.314705 28730 raft_consensus.cc:2243] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.314880 28730 raft_consensus.cc:2272] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:20.315099 28524 leader_election.cc:343] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.315279 28524 leader_election.cc:304] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.315507 28730 ts_tablet_manager.cc:1916] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.315804 28798 raft_consensus.cc:2749] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.316031 28735 raft_consensus.cc:2243] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.316223 28735 raft_consensus.cc:2272] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.317042 28735 ts_tablet_manager.cc:1916] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.317090 28739 consensus_queue.cc:1048] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260320 15:27:20.318116 28568 tablet_service.cc:1511] Processing CreateTablet for tablet da860b779a174adbb4fd6f7f7520348c (DEFAULT_TABLE table=default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.320974 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e822249154a1784ebd66f8e56abdf (DEFAULT_TABLE table=default.Table_a219d3248ce44f78a7989f8933d4d3fa [id=90afb14544844334aff0e82fa156d6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.322472 28559 tablet_service.cc:1511] Processing CreateTablet for tablet d57566e7907c4783aafce347e46b89e3 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.322880 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.323159 28267 tablet_service.cc:1511] Processing CreateTablet for tablet da957aed322f46acaf5876c567c2ff94 (DEFAULT_TABLE table=default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.323189 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet e2b08bcbee8541c998a23dbc1947796a (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
I20260320 15:27:20.323619 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da957aed322f46acaf5876c567c2ff94. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:20.324103 28239 leader_election.cc:343] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:20.326669 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da860b779a174adbb4fd6f7f7520348c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.330099 28744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.62:43951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.74.62
--webserver_port=38201
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
I20260320 15:27:20.331647 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d57566e7907c4783aafce347e46b89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.332759 28744 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:20.335484 28744 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260320 15:27:20.337571 28733 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.338231 28733 ts_tablet_manager.cc:1403] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.151s user 0.020s sys 0.022s
I20260320 15:27:20.338487 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e822249154a1784ebd66f8e56abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.339799 28733 raft_consensus.cc:359] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.340022 28733 raft_consensus.cc:385] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.340248 28735 ts_tablet_manager.cc:1929] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.340444 28735 log.cc:1199] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8e49448a973442ad88b2090527f33899
I20260320 15:27:20.340559 28733 raft_consensus.cc:740] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.340967 28733 consensus_queue.cc:260] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.343596 28733 ts_tablet_manager.cc:1434] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:20.343911 28733 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.346968 28733 tablet_bootstrap.cc:654] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.350610 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a9fa613ac9d7494a93bb8863f69d4149"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:20.350818 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0b295d872d094ac3bca1ea9effaf3081" } modify_external_catalogs: true
I20260320 15:27:20.351027 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0b295d872d094ac3bca1ea9effaf3081" } modify_external_catalogs: true
W20260320 15:27:20.351402 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a9fa613ac9d7494a93bb8863f69d4149 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.350633 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d1cf5412b72c4a58931061b1a1aba1e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.353453 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1cf5412b72c4a58931061b1a1aba1e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.355371 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_09f557af64504af69dce683cef4b27fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.356088 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09f557af64504af69dce683cef4b27fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.365087 28812 mvcc.cc:204] Tried to move back new op lower bound from 7266387722766024704 to 7266387720627396608. Current Snapshot: MvccSnapshot[applied={T|T < 7266387722766024704}]
W20260320 15:27:20.372426 28825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:20.378629 28735 ts_tablet_manager.cc:1950] T 8e49448a973442ad88b2090527f33899 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.379737 28735 tablet_replica.cc:333] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:20.380069 28735 raft_consensus.cc:2243] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.380316 28735 raft_consensus.cc:2272] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.380961 28735 ts_tablet_manager.cc:1916] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.386369 28823 tablet.cc:1723] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:20.386890 28729 ts_tablet_manager.cc:1929] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.388286 28824 tablet.cc:1723] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:20.393028 28729 log.cc:1199] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8e49448a973442ad88b2090527f33899
I20260320 15:27:20.393714 28729 ts_tablet_manager.cc:1950] T 8e49448a973442ad88b2090527f33899 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.394914 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8e49448a973442ad88b2090527f33899 (table default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]) successfully deleted
I20260320 15:27:20.390008 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545c8fd2d4c845adba6dbf42dfddba3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.391721 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc (DEFAULT_TABLE table=default.Table_714d170555c642ddaa667c974b68cbb7 [id=592cc526af644da5abbc2e49c7e38956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.397637 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:20.398202 28046 catalog_manager.cc:6741] ReplaceTablet: tablet e2586d40343b4365ab8a67cce2e1c6ca deleted and replaced by tablet 9dd293a68f894cbfa122d53918816c28
I20260320 15:27:20.400815 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 2548d7b8d31049519a85490772621352 (DEFAULT_TABLE table=default.Table_2f4e5acc5d2048e6abbd592ff5d537e4 [id=ba466e4f2691499c801ab1e748f61aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.401023 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.401284 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2548d7b8d31049519a85490772621352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.401530 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d1db9963c4645ae9aab72b4b22d6938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.402112 28046 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]). Expected version 1 got 0
I20260320 15:27:20.403702 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.404004 28274 tablet_service.cc:1511] Processing CreateTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.404458 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0e8f37a490a4fdfbc2a3dc56f6da297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.405856 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca with delete_type TABLET_DATA_DELETED (replaced by tablet 9dd293a68f894cbfa122d53918816c28) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.408962 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 203c619513a146f49a4825962f169f45 (DEFAULT_TABLE table=default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.409458 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 203c619513a146f49a4825962f169f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.409574 28736 tablet_bootstrap.cc:492] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.409739 28736 ts_tablet_manager.cc:1403] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.355s user 0.005s sys 0.008s
I20260320 15:27:20.410632 28736 raft_consensus.cc:359] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.410856 28736 raft_consensus.cc:385] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.410933 28736 raft_consensus.cc:740] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.411480 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.412936 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d2f34f53adf549cc97c5de72f7805087"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.413656 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2f34f53adf549cc97c5de72f7805087 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.415112 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ce377d5188904a50803bb93b251ac6aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.415788 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce377d5188904a50803bb93b251ac6aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.415951 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 with delete_type TABLET_DATA_DELETED (replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.416236 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.416460 28729 tablet_replica.cc:333] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:20.416445 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 484375918aed45a9bc93e7ba5ae3a113 with delete_type TABLET_DATA_DELETED (replaced by tablet 8f6f683638874e90b0d3c8969c702a5c) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.416613 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.417140 28753 raft_consensus.cc:493] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.417286 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b (DEFAULT_TABLE table=default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.417752 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c1dcc311a14b33a289ae2b8a485d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.401384 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c (DEFAULT_TABLE table=default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.420334 28736 consensus_queue.cc:260] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.420990 28736 ts_tablet_manager.cc:1434] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
W20260320 15:27:20.421032 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2548d7b8d31049519a85490772621352 consensus state has no leader
I20260320 15:27:20.421720 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_78dd3a47806c44f186bb967be20e8d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.422703 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78dd3a47806c44f186bb967be20e8d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.424263 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.425088 28736 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.428414 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc (DEFAULT_TABLE table=default.Table_714d170555c642ddaa667c974b68cbb7 [id=592cc526af644da5abbc2e49c7e38956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.429243 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b6165eab6b0442fa9d94ce577c1b43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.430292 28753 raft_consensus.cc:515] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.431306 28753 leader_election.cc:290] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:20.433912 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f35263efa3413882954c8c1aa39855" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.434178 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 (DEFAULT_TABLE table=default.Table_0b295d872d094ac3bca1ea9effaf3081 [id=5de6c807af97457ea392b60abf33a34b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.435590 28383 leader_election.cc:343] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.400121 28735 ts_tablet_manager.cc:1929] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.437697 28735 log.cc:1199] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:20.437988 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 7a8e822249154a1784ebd66f8e56abdf (DEFAULT_TABLE table=default.Table_a219d3248ce44f78a7989f8933d4d3fa [id=90afb14544844334aff0e82fa156d6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.437882 28736 tablet_bootstrap.cc:654] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.438503 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a8e822249154a1784ebd66f8e56abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.438570 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ada1f644b754fbe8b3790b46ec8cdf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.441025 28735 ts_tablet_manager.cc:1950] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.441401 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc (DEFAULT_TABLE table=default.Table_714d170555c642ddaa667c974b68cbb7 [id=592cc526af644da5abbc2e49c7e38956]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.441769 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 47f35263efa3413882954c8c1aa39855 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47f35263efa3413882954c8c1aa39855 already in progress: creating tablet
I20260320 15:27:20.442098 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.443022 28729 raft_consensus.cc:2243] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.443734 28729 pending_rounds.cc:70] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 2 pending ops.
I20260320 15:27:20.441636 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca with delete_type TABLET_DATA_DELETED (replaced by tablet 9dd293a68f894cbfa122d53918816c28) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.449793 28733 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.451615 28729 pending_rounds.cc:77] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7266387722766024704 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "484375918aed45a9bc93e7ba5ae3a113" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_a5b9cd3e79294e4481b46a6421326ac1" dest_uuid: "80636bb608a1431b955576e296ff63bb" new_extra_config { } }
I20260320 15:27:20.452557 28729 pending_rounds.cc:77] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387720627396608 op_type: NO_OP noop_request { }
I20260320 15:27:20.452967 28729 raft_consensus.cc:2889] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:20.453313 28729 raft_consensus.cc:2272] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.456653 28729 ts_tablet_manager.cc:1916] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:20.457638 28524 consensus_peers.cc:597] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e49448a973442ad88b2090527f33899. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:20.459909 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca with delete_type TABLET_DATA_DELETED (replaced by tablet 9dd293a68f894cbfa122d53918816c28) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:20.460227 28382 leader_election.cc:343] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 47f35263efa3413882954c8c1aa39855
I20260320 15:27:20.460315 28733 ts_tablet_manager.cc:1403] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.106s user 0.001s sys 0.005s
I20260320 15:27:20.460459 28382 leader_election.cc:304] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:20.463577 28753 raft_consensus.cc:2749] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.466490 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.467356 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.467602 28279 tablet_service.cc:1511] Processing CreateTablet for tablet d57566e7907c4783aafce347e46b89e3 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.468551 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d57566e7907c4783aafce347e46b89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.473217 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 (DEFAULT_TABLE table=default.Table_0b295d872d094ac3bca1ea9effaf3081 [id=5de6c807af97457ea392b60abf33a34b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.473731 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ada1f644b754fbe8b3790b46ec8cdf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.475328 28734 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:20.475696 28734 ts_tablet_manager.cc:1403] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.200s user 0.007s sys 0.002s
W20260320 15:27:20.476255 28524 consensus_peers.cc:597] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:20.477355 28734 raft_consensus.cc:359] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.477653 28268 tablet_service.cc:1511] Processing CreateTablet for tablet da860b779a174adbb4fd6f7f7520348c (DEFAULT_TABLE table=default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.477743 28734 raft_consensus.cc:385] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.478083 28734 raft_consensus.cc:740] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.478540 28734 consensus_queue.cc:260] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.481087 28730 ts_tablet_manager.cc:1929] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.481365 28730 log.cc:1199] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/1fe4ead926da4389b131d61804451bcf
I20260320 15:27:20.467633 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 (DEFAULT_TABLE table=default.Table_0b295d872d094ac3bca1ea9effaf3081 [id=5de6c807af97457ea392b60abf33a34b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.482162 28730 ts_tablet_manager.cc:1950] T 1fe4ead926da4389b131d61804451bcf P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.486979 28734 ts_tablet_manager.cc:1434] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260320 15:27:20.488129 28733 raft_consensus.cc:359] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.489118 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da860b779a174adbb4fd6f7f7520348c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.492074 28734 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:20.492514 28733 raft_consensus.cc:385] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.492698 28733 raft_consensus.cc:740] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.494344 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ada1f644b754fbe8b3790b46ec8cdf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.504387 28733 consensus_queue.cc:260] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.506913 28733 ts_tablet_manager.cc:1434] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.046s user 0.002s sys 0.000s
I20260320 15:27:20.510274 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca38bc990634e21a2c38dd840488f8c (DEFAULT_TABLE table=default.Table_d135759507f0411dbdc9e8a3608562d5 [id=96f573af716c4ff5bd62ec012e593f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.511056 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca38bc990634e21a2c38dd840488f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.518479 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 1fe4ead926da4389b131d61804451bcf (table default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]) successfully deleted
I20260320 15:27:20.519059 28730 tablet_replica.cc:333] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.519459 28730 raft_consensus.cc:2243] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.519807 28730 raft_consensus.cc:2272] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.520442 28735 tablet_replica.cc:333] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:20.520587 28730 ts_tablet_manager.cc:1916] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.521811 28735 raft_consensus.cc:2243] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.522403 28735 raft_consensus.cc:2272] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:20.522799 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b395977eb0444bcf8563cd8774fab40f consensus state has no leader
I20260320 15:27:20.523375 28735 ts_tablet_manager.cc:1916] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.528509 28733 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:20.528448 28041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20260320 15:27:20.530313 28733 tablet_bootstrap.cc:654] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.535580 28735 ts_tablet_manager.cc:1929] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:20.536684 28735 log.cc:1199] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/484375918aed45a9bc93e7ba5ae3a113
I20260320 15:27:20.537446 28735 ts_tablet_manager.cc:1950] T 484375918aed45a9bc93e7ba5ae3a113 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.539247 28735 tablet_replica.cc:333] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:20.540840 28735 raft_consensus.cc:2243] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:20.536417 28240 leader_election.cc:343] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 0a7c92a1f4894df586cbb1647ed242a3
I20260320 15:27:20.541440 28240 leader_election.cc:304] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:20.541847 28240 raft_consensus.cc:2686] T 0a7c92a1f4894df586cbb1647ed242a3 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:20.542665 28736 tablet_bootstrap.cc:492] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.542848 28733 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.542837 28736 ts_tablet_manager.cc:1403] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.119s user 0.007s sys 0.003s
I20260320 15:27:20.542991 28733 ts_tablet_manager.cc:1403] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260320 15:27:20.543953 28733 raft_consensus.cc:359] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.544163 28733 raft_consensus.cc:385] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.544318 28733 raft_consensus.cc:740] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.544276 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca38bc990634e21a2c38dd840488f8c (DEFAULT_TABLE table=default.Table_d135759507f0411dbdc9e8a3608562d5 [id=96f573af716c4ff5bd62ec012e593f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.544595 28733 consensus_queue.cc:260] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.544698 28735 pending_rounds.cc:70] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:20.544749 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca38bc990634e21a2c38dd840488f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.544780 28735 pending_rounds.cc:77] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387720091533312 op_type: NO_OP noop_request { }
I20260320 15:27:20.544912 28735 raft_consensus.cc:2889] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:20.545023 28735 raft_consensus.cc:2272] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.546584 28733 ts_tablet_manager.cc:1434] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:20.547804 28733 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.551990 28736 raft_consensus.cc:359] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.552284 28736 raft_consensus.cc:385] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.552441 28736 raft_consensus.cc:740] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.552572 28733 tablet_bootstrap.cc:654] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.552829 28736 consensus_queue.cc:260] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.554019 28730 ts_tablet_manager.cc:1929] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.554188 28730 log.cc:1199] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:20.554548 28730 ts_tablet_manager.cc:1950] T fa22d207ad8a4555aa9573b15fa1fe1a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.554951 28736 ts_tablet_manager.cc:1434] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260320 15:27:20.555287 28735 ts_tablet_manager.cc:1916] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.555961 28736 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.557850 28736 tablet_bootstrap.cc:654] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:20.559406 28524 consensus_peers.cc:597] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 484375918aed45a9bc93e7ba5ae3a113. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:20.559861 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet fa22d207ad8a4555aa9573b15fa1fe1a (table default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]) successfully deleted
I20260320 15:27:20.560287 28730 tablet_replica.cc:333] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.560496 28730 raft_consensus.cc:2243] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.560690 28730 raft_consensus.cc:2272] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.561239 28730 ts_tablet_manager.cc:1916] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.567633 28053 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]). Expected version 1 got 0
I20260320 15:27:20.568836 28733 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.569036 28733 ts_tablet_manager.cc:1403] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20260320 15:27:20.570363 28733 raft_consensus.cc:359] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.570762 28733 raft_consensus.cc:385] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.570864 28733 raft_consensus.cc:740] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.571159 28733 consensus_queue.cc:260] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.571894 28733 ts_tablet_manager.cc:1434] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:20.572242 28733 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.568789 28735 ts_tablet_manager.cc:1929] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:20.574318 28733 tablet_bootstrap.cc:654] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.574366 28735 log.cc:1199] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2ef9ab0a1f984e528f0fd7f7ce7a79db
I20260320 15:27:20.575374 28735 ts_tablet_manager.cc:1950] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.578539 28730 ts_tablet_manager.cc:1929] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.578775 28730 log.cc:1199] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/e7e9568395c44af2912efb175fc1704a
I20260320 15:27:20.579293 28730 ts_tablet_manager.cc:1950] T e7e9568395c44af2912efb175fc1704a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.580870 28736 tablet_bootstrap.cc:492] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.581079 28736 ts_tablet_manager.cc:1403] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.001s
I20260320 15:27:20.581346 28730 tablet_replica.cc:333] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.581610 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet e7e9568395c44af2912efb175fc1704a (table default.Table_0c2bc058015240478a72fe93a91a9727 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:20.581895 28736 raft_consensus.cc:359] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.582113 28736 raft_consensus.cc:385] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.582190 28736 raft_consensus.cc:740] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.582448 28736 consensus_queue.cc:260] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.583288 28736 ts_tablet_manager.cc:1434] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:20.583554 28730 raft_consensus.cc:2243] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.583683 28736 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.583770 28730 raft_consensus.cc:2272] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.584429 28730 ts_tablet_manager.cc:1916] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.585846 28736 tablet_bootstrap.cc:654] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.592895 28730 ts_tablet_manager.cc:1929] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.593117 28730 log.cc:1199] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/dfaf72b4f25c417cafd2f53db42b3315
I20260320 15:27:20.593601 28730 ts_tablet_manager.cc:1950] T dfaf72b4f25c417cafd2f53db42b3315 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.594816 28730 tablet_replica.cc:333] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.595041 28730 raft_consensus.cc:2243] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.595219 28730 raft_consensus.cc:2272] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.596016 28730 ts_tablet_manager.cc:1916] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.603454 28730 ts_tablet_manager.cc:1929] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.603649 28730 log.cc:1199] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:20.604048 28730 ts_tablet_manager.cc:1950] T 9248f0fee2fd446bb8c53891f6e7ba6e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.605229 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9248f0fee2fd446bb8c53891f6e7ba6e (table default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]) successfully deleted
W20260320 15:27:20.606056 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd already in progress: creating tablet
W20260320 15:27:20.606633 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9b993c9b6ae14ae98c2239abd65333ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b993c9b6ae14ae98c2239abd65333ea already in progress: creating tablet
W20260320 15:27:20.607208 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 3038fd8c34ee48278404acfec3e0b8d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3038fd8c34ee48278404acfec3e0b8d8 already in progress: creating tablet
I20260320 15:27:20.607523 28730 tablet_replica.cc:333] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.607810 28730 raft_consensus.cc:2243] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.608042 28730 raft_consensus.cc:2272] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.608696 28730 ts_tablet_manager.cc:1916] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.611212 28736 tablet_bootstrap.cc:492] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.612056 28736 ts_tablet_manager.cc:1403] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.000s
I20260320 15:27:20.615825 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca38bc990634e21a2c38dd840488f8c (DEFAULT_TABLE table=default.Table_d135759507f0411dbdc9e8a3608562d5 [id=96f573af716c4ff5bd62ec012e593f9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.616358 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca38bc990634e21a2c38dd840488f8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.619311 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.619827 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 557d8be0e4bf42239b2bb4585e7d0f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.619195 28736 raft_consensus.cc:359] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.620591 28736 raft_consensus.cc:385] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.621265 28736 raft_consensus.cc:740] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.622767 28736 consensus_queue.cc:260] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.623906 28567 tablet_service.cc:1511] Processing CreateTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b (DEFAULT_TABLE table=default.Table_764f1d7877d24d50b17c6d27527fdd6d [id=cb3587c86ae2454fadf5647b0e3f92eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.624341 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f591c45d9d73442ab1f0144ddfc8c48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.627108 28736 ts_tablet_manager.cc:1434] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.014s user 0.002s sys 0.002s
I20260320 15:27:20.629630 28736 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:20.637894 28041 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.62:43951, user_credentials={real_user=slave}} blocked reactor thread for 109419us
I20260320 15:27:20.639849 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.640333 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 557d8be0e4bf42239b2bb4585e7d0f06. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:20.642104 28524 consensus_peers.cc:597] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e49448a973442ad88b2090527f33899. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:20.639459 28268 tablet_service.cc:1511] Processing CreateTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b (DEFAULT_TABLE table=default.Table_764f1d7877d24d50b17c6d27527fdd6d [id=cb3587c86ae2454fadf5647b0e3f92eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.648000 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f591c45d9d73442ab1f0144ddfc8c48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.652289 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 6b6165eab6b0442fa9d94ce577c1b43c
I20260320 15:27:20.653175 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_20ceb59162bc456f86478ad67ada6956"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:20.654820 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:20.655355 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.656800 28091 master_runner.cc:186] Error getting master registration for 127.27.74.62:43951: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111),
I20260320 15:27:20.656847 28744 server_base.cc:1061] running on GCE node
I20260320 15:27:20.656965 28091 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260320 15:27:20.657629 28730 ts_tablet_manager.cc:1929] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.657829 28730 log.cc:1199] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/351a4cc8692a472d9bc5f3dbcc5ee172
I20260320 15:27:20.658281 28730 ts_tablet_manager.cc:1950] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:20.658844 28839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260320 15:27:20.659317 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:20.660640 28744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:20.662271 28730 tablet_replica.cc:333] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.662751 28730 raft_consensus.cc:2243] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:20.663579 28730 pending_rounds.cc:70] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:20.663815 28730 pending_rounds.cc:77] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387720481394688 op_type: NO_OP noop_request { }
I20260320 15:27:20.664144 28730 raft_consensus.cc:2889] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:20.665011 28730 raft_consensus.cc:2272] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.640719 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0a7c92a1f4894df586cbb1647ed242a3 (table default.Table_54061406504947f48bb5bf99514ead4a [id=5d6390bc91444075a52ea7ec4abce623]) successfully deleted
W20260320 15:27:20.667785 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20ceb59162bc456f86478ad67ada6956 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:20.668155 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fa22d207ad8a4555aa9573b15fa1fe1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa22d207ad8a4555aa9573b15fa1fe1a already in progress: creating tablet
W20260320 15:27:20.668563 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1fe4ead926da4389b131d61804451bcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fe4ead926da4389b131d61804451bcf already in progress: creating tablet
W20260320 15:27:20.668913 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4902ffdcdbc54d5aa9c71f6cd239c69a already in progress: creating tablet
W20260320 15:27:20.669212 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e7e9568395c44af2912efb175fc1704a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7e9568395c44af2912efb175fc1704a already in progress: creating tablet
W20260320 15:27:20.669504 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 22b7f757ebdf456eb81085e5be54da4b already in progress: creating tablet
W20260320 15:27:20.669895 28041 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0ab709738a3f471da773fd9b523f7fbb because the tablet was not found. No further retry: Not found: Tablet not found: 0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:20.669817 28730 ts_tablet_manager.cc:1916] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:20.670262 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8e3284c73cfa4dfba65a88a7dd243190 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e3284c73cfa4dfba65a88a7dd243190 already in progress: creating tablet
I20260320 15:27:20.671144 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.672083 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 4b4040a0214a46b580b2ac2de59243ce (DEFAULT_TABLE table=default.Table_47ac32f0123e4e8e87fff99efa39eca0 [id=18c937cd7a8344e5a9c283fb8d303f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.672521 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4040a0214a46b580b2ac2de59243ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.673354 28736 tablet_bootstrap.cc:654] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.674911 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 8f6f683638874e90b0d3c8969c702a5c (DEFAULT_TABLE table=default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.675428 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f6f683638874e90b0d3c8969c702a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.675890 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2339ff2c474a288876f32b85cecc72 (DEFAULT_TABLE table=default.Table_d4066b167fcb4656b497ac2ef62d5366 [id=cbccbfa505024b1b89d6c80ef592a812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.676268 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2339ff2c474a288876f32b85cecc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.684924 28733 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.685245 28733 ts_tablet_manager.cc:1403] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.113s user 0.006s sys 0.011s
I20260320 15:27:20.686338 28733 raft_consensus.cc:359] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.686666 28733 raft_consensus.cc:385] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.686839 28733 raft_consensus.cc:740] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.687207 28733 consensus_queue.cc:260] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.693084 28733 ts_tablet_manager.cc:1434] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:20.693190 28736 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.693464 28736 ts_tablet_manager.cc:1403] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.000s
I20260320 15:27:20.694617 28736 raft_consensus.cc:359] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.694931 28736 raft_consensus.cc:385] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.695046 28736 raft_consensus.cc:740] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.695472 28736 consensus_queue.cc:260] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.696837 28730 ts_tablet_manager.cc:1929] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:20.697688 28730 log.cc:1199] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8e49448a973442ad88b2090527f33899
I20260320 15:27:20.698133 28733 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.699159 28739 raft_consensus.cc:493] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.699348 28739 raft_consensus.cc:515] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.700137 28739 leader_election.cc:290] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.700093 28730 ts_tablet_manager.cc:1950] T 8e49448a973442ad88b2090527f33899 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.701278 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee4aafbcc4e46bb9dbb0460a8575282" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.701694 28736 ts_tablet_manager.cc:1434] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
W20260320 15:27:20.701920 28524 leader_election.cc:343] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.704622 28733 tablet_bootstrap.cc:654] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.705423 28730 tablet_replica.cc:333] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.706256 28730 raft_consensus.cc:2243] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.706451 28736 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.707062 28730 raft_consensus.cc:2272] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.709054 28736 tablet_bootstrap.cc:654] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.711185 28730 ts_tablet_manager.cc:1916] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.713297 28729 ts_tablet_manager.cc:1929] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260320 15:27:20.714437 28240 leader_election.cc:343] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 88a005b1c2b54a2fa8515f7a6b165ea8
I20260320 15:27:20.714669 28240 leader_election.cc:304] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:20.714934 28240 raft_consensus.cc:2686] T 88a005b1c2b54a2fa8515f7a6b165ea8 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:20.716248 28729 log.cc:1199] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/484375918aed45a9bc93e7ba5ae3a113
I20260320 15:27:20.716826 28729 ts_tablet_manager.cc:1950] T 484375918aed45a9bc93e7ba5ae3a113 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.718448 28729 tablet_replica.cc:333] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:20.718662 28729 raft_consensus.cc:2243] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.718880 28729 raft_consensus.cc:2272] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.719520 28729 ts_tablet_manager.cc:1916] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.720624 28425 tablet_service.cc:1511] Processing CreateTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b (DEFAULT_TABLE table=default.Table_764f1d7877d24d50b17c6d27527fdd6d [id=cb3587c86ae2454fadf5647b0e3f92eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.721113 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f591c45d9d73442ab1f0144ddfc8c48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.723618 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee4aafbcc4e46bb9dbb0460a8575282" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:20.724428 28736 tablet_bootstrap.cc:492] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.724707 28736 ts_tablet_manager.cc:1403] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.003s
I20260320 15:27:20.725018 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 (DEFAULT_TABLE table=default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.725751 28524 leader_election.cc:343] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.725904 28524 leader_election.cc:304] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.725905 28736 raft_consensus.cc:359] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:20.726131 28524 raft_consensus.cc:2686] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:20.726172 28736 raft_consensus.cc:385] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.726277 28736 raft_consensus.cc:740] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.726662 28736 consensus_queue.cc:260] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.727758 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 557d8be0e4bf42239b2bb4585e7d0f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.732337 28736 ts_tablet_manager.cc:1434] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260320 15:27:20.737187 28736 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.738607 28730 ts_tablet_manager.cc:1929] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.739244 28730 log.cc:1199] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0ee4aafbcc4e46bb9dbb0460a8575282
I20260320 15:27:20.739977 28736 tablet_bootstrap.cc:654] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.741039 28730 ts_tablet_manager.cc:1950] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.745357 28730 tablet_replica.cc:333] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.746277 28730 raft_consensus.cc:2243] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.746933 28730 raft_consensus.cc:2272] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.749399 28730 ts_tablet_manager.cc:1916] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.750325 28734 tablet_bootstrap.cc:654] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.758569 28729 ts_tablet_manager.cc:1929] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.671061 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_05e8ba9bd1c24a1694c6b7ef150e9624"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.765717 28041 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 95446.2us
W20260320 15:27:20.766062 28041 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9248f0fee2fd446bb8c53891f6e7ba6e because the tablet was not found. No further retry: Not found: Tablet not found: 9248f0fee2fd446bb8c53891f6e7ba6e
W20260320 15:27:20.766325 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05e8ba9bd1c24a1694c6b7ef150e9624 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:20.767176 28041 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd because the tablet was not found. No further retry: Not found: Tablet not found: ae4ec6ec99df4fb5b1ebec5b1437c8fd
W20260320 15:27:20.767447 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3038fd8c34ee48278404acfec3e0b8d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3038fd8c34ee48278404acfec3e0b8d8 already in progress: creating tablet
I20260320 15:27:20.671597 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c12e189dde7543a282d5237cf3200a0c" } new_table_name: "default.Table_35e50a930cbd465b95ca84b3cea68490" modify_external_catalogs: true
I20260320 15:27:20.704514 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8e49448a973442ad88b2090527f33899 (table default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]) successfully deleted
W20260320 15:27:20.768172 28040 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 63691.4us
I20260320 15:27:20.768908 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (table default.Table_15f5a33964a94937ada1a7df25fda363 [id=41bf3a11683345ff9162410e671ee6ff]) successfully deleted
W20260320 15:27:20.710958 28038 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet df5ffaedeaf94def93d93a538c42e3bd (table default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet df5ffaedeaf94def93d93a538c42e3bd consensus state has no leader
I20260320 15:27:20.770032 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (DEFAULT_TABLE table=default.Table_9c0b3e51cc4042fab00ee1662295787e [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.770530 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.779255 28733 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.779475 28733 ts_tablet_manager.cc:1403] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.082s user 0.004s sys 0.000s
I20260320 15:27:20.780381 28733 raft_consensus.cc:359] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.780632 28733 raft_consensus.cc:385] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.780721 28733 raft_consensus.cc:740] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.780992 28733 consensus_queue.cc:260] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.787199 28733 ts_tablet_manager.cc:1434] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260320 15:27:20.787673 28733 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.789747 28733 tablet_bootstrap.cc:654] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.791929 28729 log.cc:1199] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2ef9ab0a1f984e528f0fd7f7ce7a79db
I20260320 15:27:20.792539 28729 ts_tablet_manager.cc:1950] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.791793 28734 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:20.792893 28734 ts_tablet_manager.cc:1403] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.302s user 0.006s sys 0.000s
I20260320 15:27:20.793833 28734 raft_consensus.cc:359] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.800851 28734 raft_consensus.cc:385] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.801029 28734 raft_consensus.cc:740] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.801486 28734 consensus_queue.cc:260] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.802312 28734 ts_tablet_manager.cc:1434] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:20.802739 28734 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:20.804804 28734 tablet_bootstrap.cc:654] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.806921 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8e49448a973442ad88b2090527f33899 (table default.Table_ebc7ffacd75e49a4b2ed94aafff57936 [id=69c62139e3644de2a7e8480a5ea58211]) successfully deleted
I20260320 15:27:20.807575 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 88a005b1c2b54a2fa8515f7a6b165ea8 (table default.Table_0c2bc058015240478a72fe93a91a9727 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:20.807924 28730 ts_tablet_manager.cc:1929] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:20.808153 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9b993c9b6ae14ae98c2239abd65333ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9b993c9b6ae14ae98c2239abd65333ea already in progress: creating tablet
I20260320 15:27:20.808772 28730 log.cc:1199] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/19c939f8c30943fe8f0619b7dcacc7e0
W20260320 15:27:20.809643 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ee4aafbcc4e46bb9dbb0460a8575282 already in progress: creating tablet
W20260320 15:27:20.810066 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 19c939f8c30943fe8f0619b7dcacc7e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19c939f8c30943fe8f0619b7dcacc7e0 already in progress: creating tablet
I20260320 15:27:20.810497 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]) successfully deleted
W20260320 15:27:20.810765 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 801b3b0ba2344cbe8d89111e17e546ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 801b3b0ba2344cbe8d89111e17e546ee already in progress: creating tablet
I20260320 15:27:20.811061 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]) successfully deleted
W20260320 15:27:20.811318 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e2586d40343b4365ab8a67cce2e1c6ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2586d40343b4365ab8a67cce2e1c6ca already in progress: creating tablet
W20260320 15:27:20.811496 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc already in progress: creating tablet
I20260320 15:27:20.812443 28730 ts_tablet_manager.cc:1950] T 19c939f8c30943fe8f0619b7dcacc7e0 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.814750 28729 tablet_replica.cc:333] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:20.815047 28729 raft_consensus.cc:2243] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.815274 28729 raft_consensus.cc:2272] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.815994 28729 ts_tablet_manager.cc:1916] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.818328 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 19c939f8c30943fe8f0619b7dcacc7e0 (table default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]) successfully deleted
I20260320 15:27:20.819386 28730 tablet_replica.cc:333] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.820698 28730 raft_consensus.cc:2243] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:20.822577 28730 pending_rounds.cc:70] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:20.822829 28730 pending_rounds.cc:77] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7266387724515262464 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "484375918aed45a9bc93e7ba5ae3a113" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_a5b9cd3e79294e4481b46a6421326ac1" dest_uuid: "80636bb608a1431b955576e296ff63bb" new_extra_config { } }
I20260320 15:27:20.823889 28744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260320 15:27:20.824525 28730 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:51060: Aborted: Op aborted
W20260320 15:27:20.825969 28040 catalog_manager.cc:5081] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): alter failed for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]): Aborted: Op aborted
I20260320 15:27:20.828336 28730 raft_consensus.cc:2272] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:20.828974 28806 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:51060: Illegal state: RaftConsensus is not running: State = Stopped
I20260320 15:27:20.830701 28730 ts_tablet_manager.cc:1916] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.841320 28817 raft_consensus.cc:493] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.841589 28817 raft_consensus.cc:515] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.842748 28817 leader_election.cc:290] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:20.845582 28239 leader_election.cc:343] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: cec531361625474f93daa8453b39f944
W20260320 15:27:20.846073 28240 leader_election.cc:343] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: cec531361625474f93daa8453b39f944
I20260320 15:27:20.846212 28240 leader_election.cc:304] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.846946 28817 raft_consensus.cc:2749] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.846963 28730 ts_tablet_manager.cc:1929] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260320 15:27:20.847160 28730 log.cc:1199] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/484375918aed45a9bc93e7ba5ae3a113
I20260320 15:27:20.849718 28730 ts_tablet_manager.cc:1950] T 484375918aed45a9bc93e7ba5ae3a113 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.849653 28739 raft_consensus.cc:493] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.850046 28739 raft_consensus.cc:515] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.851020 28730 tablet_replica.cc:333] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.851290 28730 raft_consensus.cc:2243] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.851471 28730 raft_consensus.cc:2272] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.851922 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9c0e5be4f98e48b8aecebbf9db56e138" } new_table_name: "default.Table_726c3589aba84426980572e969866fa3" modify_external_catalogs: true
I20260320 15:27:20.852070 28730 ts_tablet_manager.cc:1916] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.852300 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab709738a3f471da773fd9b523f7fbb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.852906 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ab709738a3f471da773fd9b523f7fbb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:20.854307 28524 leader_election.cc:343] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.854974 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]) successfully deleted
I20260320 15:27:20.855408 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet b0e8f37a490a4fdfbc2a3dc56f6da297 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:20.855578 28524 leader_election.cc:343] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.855715 28524 leader_election.cc:304] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.855986 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_47ac32f0123e4e8e87fff99efa39eca0" } new_table_name: "default.Table_e081f8a109224b479a6e7aec2d18356b" modify_external_catalogs: true
I20260320 15:27:20.856125 28798 raft_consensus.cc:2749] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:20.856878 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1268b25a1c99404aa77e5b1bcc415f48 consensus state has no leader
I20260320 15:27:20.856945 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d4066b167fcb4656b497ac2ef62d5366" } modify_external_catalogs: true
I20260320 15:27:20.857151 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d4066b167fcb4656b497ac2ef62d5366" } modify_external_catalogs: true
I20260320 15:27:20.857177 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e976aa12e3ab47bd8d42318871768eba" } modify_external_catalogs: true
I20260320 15:27:20.857230 28054 master_service.cc:954] ReplaceTablet: received request to replace tablet df5ffaedeaf94def93d93a538c42e3bd from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.857313 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e976aa12e3ab47bd8d42318871768eba" } modify_external_catalogs: true
I20260320 15:27:20.856942 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c4feb7323b9748c9b7f01574beecc884" } modify_external_catalogs: true
I20260320 15:27:20.858048 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c4feb7323b9748c9b7f01574beecc884" } modify_external_catalogs: true
I20260320 15:27:20.859261 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a894581517cb4471a20cffa8c3ec26f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:20.860031 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a894581517cb4471a20cffa8c3ec26f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:20.860867 28045 master_service.cc:954] ReplaceTablet: received request to replace tablet 81a1e0957439458083e3a86ac2f01b1e from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.851038 28739 leader_election.cc:290] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.869277 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:20.870970 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9dd293a68f894cbfa122d53918816c28 has no consensus state
I20260320 15:27:20.871280 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 484375918aed45a9bc93e7ba5ae3a113 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.871763 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 3038fd8c34ee48278404acfec3e0b8d8 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:20.872339 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b61482084570428fadb57b2429b92e99" } new_table_name: "default.Table_04b04acd60a042188e87b5d447398dce" modify_external_catalogs: true
I20260320 15:27:20.873790 28730 ts_tablet_manager.cc:1929] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.873914 28730 log.cc:1199] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/801b3b0ba2344cbe8d89111e17e546ee
I20260320 15:27:20.874329 28730 ts_tablet_manager.cc:1950] T 801b3b0ba2344cbe8d89111e17e546ee P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.875406 28730 tablet_replica.cc:333] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.875522 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 801b3b0ba2344cbe8d89111e17e546ee (table default.Table_04f3937bd89341f98bf5a89d8b00de07 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]) successfully deleted
I20260320 15:27:20.875861 28730 raft_consensus.cc:2243] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.876116 28730 raft_consensus.cc:2272] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.876791 28730 ts_tablet_manager.cc:1916] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.878427 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:20.879148 28054 catalog_manager.cc:6741] ReplaceTablet: tablet df5ffaedeaf94def93d93a538c42e3bd deleted and replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:20.879359 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 28c1dcc311a14b33a289ae2b8a485d2b
I20260320 15:27:20.879714 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.880347 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.879143 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ea3849a556d94649a3d029b4ac41fcf4
I20260320 15:27:20.881801 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.885236 28736 tablet_bootstrap.cc:492] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:20.885251 28054 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]). Expected version 1 got 0
I20260320 15:27:20.885545 28736 ts_tablet_manager.cc:1403] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.148s user 0.044s sys 0.060s
I20260320 15:27:20.886039 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:20.886543 28052 catalog_manager.cc:6741] ReplaceTablet: tablet b0e8f37a490a4fdfbc2a3dc56f6da297 deleted and replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a
I20260320 15:27:20.887287 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.886468 28736 raft_consensus.cc:359] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.887758 28736 raft_consensus.cc:385] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.887862 28736 raft_consensus.cc:740] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.887866 28744 hybrid_clock.cc:648] HybridClock initialized: now 1774020440887718 us; error 151 us; skew 500 ppm
I20260320 15:27:20.888120 28736 consensus_queue.cc:260] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.888428 28744 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:20.888603 28734 tablet_bootstrap.cc:492] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:20.888680 28736 ts_tablet_manager.cc:1434] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260320 15:27:20.888912 28734 ts_tablet_manager.cc:1403] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.002s
I20260320 15:27:20.889048 28736 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:20.890787 28734 raft_consensus.cc:359] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.891042 28736 tablet_bootstrap.cc:654] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.893152 28730 ts_tablet_manager.cc:1929] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.893374 28734 raft_consensus.cc:385] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.893657 28734 raft_consensus.cc:740] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.894016 28734 consensus_queue.cc:260] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.898514 28734 ts_tablet_manager.cc:1434] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260320 15:27:20.899029 28734 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:20.899358 28730 log.cc:1199] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2ef9ab0a1f984e528f0fd7f7ce7a79db
I20260320 15:27:20.902273 28730 ts_tablet_manager.cc:1950] T 2ef9ab0a1f984e528f0fd7f7ce7a79db P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.902621 28729 ts_tablet_manager.cc:1929] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.902786 28729 log.cc:1199] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/47f35263efa3413882954c8c1aa39855
I20260320 15:27:20.903275 28729 ts_tablet_manager.cc:1950] T 47f35263efa3413882954c8c1aa39855 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:20.905061 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]) successfully deleted
I20260320 15:27:20.896703 28733 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.905439 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet cec531361625474f93daa8453b39f944 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.905825 28735 tablet_replica.cc:333] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:20.905809 28733 ts_tablet_manager.cc:1403] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.118s user 0.004s sys 0.000s
I20260320 15:27:20.905923 28744 webserver.cc:492] Webserver started at http://127.27.74.62:38201/ using document root <none> and password file <none>
I20260320 15:27:20.906054 28735 raft_consensus.cc:2243] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.906368 28735 raft_consensus.cc:2272] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.907131 28744 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:20.907225 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.907337 28744 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:20.907413 28735 ts_tablet_manager.cc:1916] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.907606 28733 raft_consensus.cc:359] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.907835 28733 raft_consensus.cc:385] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.907949 28733 raft_consensus.cc:740] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.908048 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:20.908922 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc already in progress: creating tablet
I20260320 15:27:20.909399 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:20.909812 28734 tablet_bootstrap.cc:654] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.911438 28739 raft_consensus.cc:493] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.911746 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:20.911641 28739 raft_consensus.cc:515] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.912443 28739 leader_election.cc:290] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.912590 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 4b4040a0214a46b580b2ac2de59243ce (DEFAULT_TABLE table=default.Table_47ac32f0123e4e8e87fff99efa39eca0 [id=18c937cd7a8344e5a9c283fb8d303f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.913064 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4040a0214a46b580b2ac2de59243ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.915891 28735 ts_tablet_manager.cc:1929] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.916085 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:20.916426 28735 log.cc:1199] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/cec531361625474f93daa8453b39f944
I20260320 15:27:20.916656 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:20.916782 28524 leader_election.cc:343] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:20.918216 28524 leader_election.cc:343] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.918416 28524 leader_election.cc:304] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.918812 28735 ts_tablet_manager.cc:1950] T cec531361625474f93daa8453b39f944 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:20.918999 28739 raft_consensus.cc:2749] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.923048 28739 raft_consensus.cc:493] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.923269 28739 raft_consensus.cc:515] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.923969 28739 leader_election.cc:290] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.924759 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:20.924966 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:20.925410 28524 leader_election.cc:343] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.925719 28733 consensus_queue.cc:260] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:20.925871 28524 leader_election.cc:343] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.926034 28524 leader_election.cc:304] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.926302 28730 tablet_replica.cc:333] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:20.926510 28730 raft_consensus.cc:2243] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:20.928452 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 4b4040a0214a46b580b2ac2de59243ce (DEFAULT_TABLE table=default.Table_47ac32f0123e4e8e87fff99efa39eca0 [id=18c937cd7a8344e5a9c283fb8d303f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.928927 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b4040a0214a46b580b2ac2de59243ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.936162 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2339ff2c474a288876f32b85cecc72 (DEFAULT_TABLE table=default.Table_d4066b167fcb4656b497ac2ef62d5366 [id=cbccbfa505024b1b89d6c80ef592a812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.936511 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 0c2339ff2c474a288876f32b85cecc72 (DEFAULT_TABLE table=default.Table_d4066b167fcb4656b497ac2ef62d5366 [id=cbccbfa505024b1b89d6c80ef592a812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.936656 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2339ff2c474a288876f32b85cecc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.937067 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c2339ff2c474a288876f32b85cecc72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.939107 28739 raft_consensus.cc:2749] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.939836 28733 ts_tablet_manager.cc:1434] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.034s user 0.002s sys 0.000s
I20260320 15:27:20.940184 28733 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:20.937959 28744 fs_manager.cc:714] Time spent opening directory manager: real 0.028s user 0.010s sys 0.007s
I20260320 15:27:20.941553 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.944196 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 8f6f683638874e90b0d3c8969c702a5c (DEFAULT_TABLE table=default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:20.944341 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b4040a0214a46b580b2ac2de59243ce consensus state has no leader
I20260320 15:27:20.944691 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f6f683638874e90b0d3c8969c702a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.944732 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 8f6f683638874e90b0d3c8969c702a5c (DEFAULT_TABLE table=default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.945175 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f6f683638874e90b0d3c8969c702a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.946619 28730 raft_consensus.cc:2272] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:20.947245 28730 ts_tablet_manager.cc:1916] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:20.950913 28733 tablet_bootstrap.cc:654] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.951933 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:20.954200 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.956276 28753 raft_consensus.cc:493] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.956514 28753 raft_consensus.cc:515] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.956751 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (DEFAULT_TABLE table=default.Table_9c0b3e51cc4042fab00ee1662295787e [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.957057 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (DEFAULT_TABLE table=default.Table_9c0b3e51cc4042fab00ee1662295787e [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:20.957201 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.957216 28753 leader_election.cc:290] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:20.957504 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:20.957954 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4902ffdcdbc54d5aa9c71f6cd239c69a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:20.959623 28383 leader_election.cc:343] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.960325 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]) successfully deleted
W20260320 15:27:20.960695 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ee4aafbcc4e46bb9dbb0460a8575282 already in progress: creating tablet
W20260320 15:27:20.961057 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 19c939f8c30943fe8f0619b7dcacc7e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 19c939f8c30943fe8f0619b7dcacc7e0 already in progress: creating tablet
I20260320 15:27:20.961810 28733 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:20.962036 28733 ts_tablet_manager.cc:1403] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260320 15:27:20.962270 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 2ef9ab0a1f984e528f0fd7f7ce7a79db (table default.Table_549f77fe41f947ab8e14fbce182049f8 [id=d3714568b2244ef1ab538461b3a730f8]) successfully deleted
W20260320 15:27:20.962592 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 801b3b0ba2344cbe8d89111e17e546ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 801b3b0ba2344cbe8d89111e17e546ee already in progress: creating tablet
W20260320 15:27:20.962874 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc already in progress: creating tablet
I20260320 15:27:20.964314 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:20.964969 28045 catalog_manager.cc:6741] ReplaceTablet: tablet 81a1e0957439458083e3a86ac2f01b1e deleted and replaced by tablet 931f6f88a1d94a40b69f07216b8313c8
W20260320 15:27:20.965131 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 026bb078f3904a44820442e69e654f72 consensus state has no leader
I20260320 15:27:20.965894 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:20.963020 28733 raft_consensus.cc:359] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.968856 28733 raft_consensus.cc:385] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.969017 28733 raft_consensus.cc:740] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.969431 28733 consensus_queue.cc:260] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.972477 28733 ts_tablet_manager.cc:1434] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:20.972926 28733 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:20.974267 28382 leader_election.cc:343] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 4902ffdcdbc54d5aa9c71f6cd239c69a
I20260320 15:27:20.974454 28382 leader_election.cc:304] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:20.976410 28730 ts_tablet_manager.cc:1929] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:20.976616 28730 log.cc:1199] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:20.977093 28730 ts_tablet_manager.cc:1950] T e2586d40343b4365ab8a67cce2e1c6ca P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:20.978075 28753 raft_consensus.cc:2749] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.978339 28734 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:20.978606 28734 ts_tablet_manager.cc:1403] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.080s user 0.005s sys 0.000s
I20260320 15:27:20.979656 28734 raft_consensus.cc:359] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.979736 28817 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.979977 28734 raft_consensus.cc:385] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:20.979895 28817 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.980134 28734 raft_consensus.cc:740] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:20.980499 28734 consensus_queue.cc:260] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.980625 28817 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:20.981487 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
W20260320 15:27:20.983368 28240 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
W20260320 15:27:20.984092 28239 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:20.984341 28239 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:20.984965 28817 raft_consensus.cc:493] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.985162 28817 raft_consensus.cc:515] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.986079 28734 ts_tablet_manager.cc:1434] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:20.986760 28734 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:20.991932 28734 tablet_bootstrap.cc:654] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:20.993080 28817 leader_election.cc:290] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:20.993357 28817 raft_consensus.cc:2749] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:20.993856 28739 raft_consensus.cc:493] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:20.994020 28739 raft_consensus.cc:515] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:20.994720 28739 leader_election.cc:290] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:20.995662 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ee8b3598384569bbe3d63abb3eaed9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:20.996371 28524 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:20.997437 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe4ead926da4389b131d61804451bcf" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:20.997764 28444 raft_consensus.cc:2468] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:20.998488 28240 leader_election.cc:304] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:20.998886 28817 raft_consensus.cc:2804] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:20.999171 28817 raft_consensus.cc:493] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:20.999284 28817 raft_consensus.cc:3060] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:21.004141 28559 tablet_service.cc:1511] Processing CreateTablet for tablet d547c032332d4d109222b4743fa28561 (DEFAULT_TABLE table=default.Table_05e8ba9bd1c24a1694c6b7ef150e9624 [id=c4809856807c41debfddd0116dc7aeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.004698 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d547c032332d4d109222b4743fa28561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.008482 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 993e2fb1561f4317be85e9b5e5816edc (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.008426 28564 tablet_service.cc:1511] Processing CreateTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a (DEFAULT_TABLE table=default.Table_d1cf5412b72c4a58931061b1a1aba1e6 [id=fc92586ccab74ae3960a1eacc8cbac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.008426 28568 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3ee3c611d41b483bd35f868042d05 (DEFAULT_TABLE table=default.Table_78dd3a47806c44f186bb967be20e8d9f [id=79ab13b2c3fa42b4b744700487fd8b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.009197 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993e2fb1561f4317be85e9b5e5816edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.009500 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3ee3c611d41b483bd35f868042d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.011473 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_15f5a33964a94937ada1a7df25fda363" } new_table_name: "default.Table_5abb1b4446804b0eb71ccecaa7dffaf3" modify_external_catalogs: true
I20260320 15:27:21.012012 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 093ae84740d5468188b4a56fa16ded9c (DEFAULT_TABLE table=default.Table_20ceb59162bc456f86478ad67ada6956 [id=e79e8aae444f489eb30ffa5b6237e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.012470 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 093ae84740d5468188b4a56fa16ded9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.012696 28817 raft_consensus.cc:515] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.013391 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcec6175c1ca49f5b682a29d5c1e235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.016371 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.016716 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet e2b08bcbee8541c998a23dbc1947796a with delete_type TABLET_DATA_DELETED (replaced by tablet 0ab709738a3f471da773fd9b523f7fbb) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.016975 28735 tablet_replica.cc:333] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.017577 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 with delete_type TABLET_DATA_DELETED (replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.018232 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.019069 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.020083 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.020704 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.021006 28735 raft_consensus.cc:2243] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.021637 28735 raft_consensus.cc:2272] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.021903 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.022339 28817 leader_election.cc:290] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:21.022696 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fe4ead926da4389b131d61804451bcf" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:21.022948 28444 raft_consensus.cc:3060] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:21.023710 28735 ts_tablet_manager.cc:1916] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.025128 28733 tablet_bootstrap.cc:654] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.026068 28444 raft_consensus.cc:2468] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:21.026870 28856 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:21.027150 28240 leader_election.cc:304] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:21.027379 28240 raft_consensus.cc:2686] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260320 15:27:21.027892 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 6b6165eab6b0442fa9d94ce577c1b43c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b6165eab6b0442fa9d94ce577c1b43c already in progress: creating tablet
W20260320 15:27:21.029446 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ada1f644b754fbe8b3790b46ec8cdf4 already in progress: creating tablet
I20260320 15:27:21.030962 28730 tablet_replica.cc:333] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.031162 28735 ts_tablet_manager.cc:1929] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.031441 28735 log.cc:1199] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1fe4ead926da4389b131d61804451bcf
I20260320 15:27:21.031776 28730 raft_consensus.cc:2243] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.031970 28730 raft_consensus.cc:2272] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.031987 28735 ts_tablet_manager.cc:1950] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.033063 28735 tablet_replica.cc:333] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.033313 28735 raft_consensus.cc:2243] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.033578 28735 raft_consensus.cc:2272] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.034415 28735 ts_tablet_manager.cc:1916] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:21.035449 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 557d8be0e4bf42239b2bb4585e7d0f06 consensus state has no leader
I20260320 15:27:21.035753 28729 tablet_replica.cc:333] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.035969 28729 raft_consensus.cc:2243] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:21.036417 28729 pending_rounds.cc:70] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:21.036516 28729 pending_rounds.cc:77] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387723265511424 op_type: NO_OP noop_request { }
I20260320 15:27:21.036670 28729 raft_consensus.cc:2889] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:21.036787 28729 raft_consensus.cc:2272] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.037927 28736 tablet_bootstrap.cc:492] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.037895 28744 fs_manager.cc:730] Time spent opening block manager: real 0.095s user 0.018s sys 0.037s
I20260320 15:27:21.038197 28736 ts_tablet_manager.cc:1403] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.149s user 0.006s sys 0.000s
I20260320 15:27:21.038918 28729 ts_tablet_manager.cc:1916] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.038127 28744 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
uuid: "ee1caebf640d4ec0bf9b28c4f496ca18"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:21.039259 28736 raft_consensus.cc:359] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.039309 28744 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:21.039752 28736 raft_consensus.cc:385] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.039952 28736 raft_consensus.cc:740] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.040290 28736 consensus_queue.cc:260] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.041090 28730 ts_tablet_manager.cc:1916] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.045367 28729 ts_tablet_manager.cc:1929] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:21.045679 28729 log.cc:1199] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:21.046285 28729 ts_tablet_manager.cc:1950] T e2586d40343b4365ab8a67cce2e1c6ca P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.047379 28733 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.047609 28733 ts_tablet_manager.cc:1403] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.001s
I20260320 15:27:21.048158 28729 tablet_replica.cc:333] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.048298 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.048419 28729 raft_consensus.cc:2243] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.048478 28733 raft_consensus.cc:359] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.048717 28729 raft_consensus.cc:2272] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.048770 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.049059 28733 raft_consensus.cc:385] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.049175 28733 raft_consensus.cc:740] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.049528 28733 consensus_queue.cc:260] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.049831 28729 ts_tablet_manager.cc:1916] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.050212 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.051273 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fe4ead926da4389b131d61804451bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.052094 28733 ts_tablet_manager.cc:1434] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:21.055845 28733 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:21.057956 28734 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.058171 28734 ts_tablet_manager.cc:1403] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.072s user 0.006s sys 0.001s
I20260320 15:27:21.059110 28734 raft_consensus.cc:359] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.059479 28734 raft_consensus.cc:385] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.041558 28735 ts_tablet_manager.cc:1929] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.059623 28020 master_runner.cc:186] Error getting master registration for 127.27.74.62:43951: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111),
I20260320 15:27:21.059729 28735 log.cc:1199] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/e2b08bcbee8541c998a23dbc1947796a
I20260320 15:27:21.059798 28020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260320 15:27:21.059741 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 2165371f4b5448c7a2625926082ee4c6 (DEFAULT_TABLE table=default.Table_a9fa613ac9d7494a93bb8863f69d4149 [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:21.046204 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 62130.1us
I20260320 15:27:21.060171 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2165371f4b5448c7a2625926082ee4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.060285 28735 ts_tablet_manager.cc:1950] T e2b08bcbee8541c998a23dbc1947796a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.049556 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.060627 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ee8b3598384569bbe3d63abb3eaed9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:21.057624 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet e7e9568395c44af2912efb175fc1704a with delete_type TABLET_DATA_DELETED (replaced by tablet 9dbd3b0e92a744dcbc132037c67303ab) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:21.061735 28524 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.061920 28524 leader_election.cc:304] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.062157 28736 ts_tablet_manager.cc:1434] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260320 15:27:21.062379 28739 raft_consensus.cc:2749] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.062474 28736 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.064599 28735 tablet_replica.cc:333] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.064841 28735 raft_consensus.cc:2243] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.065052 28735 raft_consensus.cc:2272] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.059283 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.065310 28734 raft_consensus.cc:740] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.065660 28734 consensus_queue.cc:260] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.066232 28735 ts_tablet_manager.cc:1916] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.059368 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.067358 28734 ts_tablet_manager.cc:1434] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260320 15:27:21.068439 28733 tablet_bootstrap.cc:654] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.068796 28734 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.071727 28734 tablet_bootstrap.cc:654] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.077118 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.079197 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.079849 28425 tablet_service.cc:1511] Processing CreateTablet for tablet d547c032332d4d109222b4743fa28561 (DEFAULT_TABLE table=default.Table_05e8ba9bd1c24a1694c6b7ef150e9624 [id=c4809856807c41debfddd0116dc7aeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.080467 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d547c032332d4d109222b4743fa28561. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.082840 28239 leader_election.cc:343] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 1fe4ead926da4389b131d61804451bcf
W20260320 15:27:21.083271 28239 leader_election.cc:343] T 1fe4ead926da4389b131d61804451bcf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 1fe4ead926da4389b131d61804451bcf
I20260320 15:27:21.088518 28267 tablet_service.cc:1511] Processing CreateTablet for tablet d547c032332d4d109222b4743fa28561 (DEFAULT_TABLE table=default.Table_05e8ba9bd1c24a1694c6b7ef150e9624 [id=c4809856807c41debfddd0116dc7aeb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.088828 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.089151 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d547c032332d4d109222b4743fa28561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.090466 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 66a9ddea49e641da881fba339ddef549 (DEFAULT_TABLE table=default.Table_dfa57bec1c394719acd3a8098ed1905d [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.090940 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a9ddea49e641da881fba339ddef549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.091161 28729 ts_tablet_manager.cc:1929] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.091295 28729 log.cc:1199] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/dfaf72b4f25c417cafd2f53db42b3315
I20260320 15:27:21.091709 28729 ts_tablet_manager.cc:1950] T dfaf72b4f25c417cafd2f53db42b3315 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.093204 28729 tablet_replica.cc:333] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.093436 28729 raft_consensus.cc:2243] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.093623 28729 raft_consensus.cc:2272] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.094208 28729 ts_tablet_manager.cc:1916] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.100566 28734 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.100610 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd293a68f894cbfa122d53918816c28 (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.100654 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 993e2fb1561f4317be85e9b5e5816edc (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:21.100876 28040 catalog_manager.cc:5075] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): alter failed for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_a5b9cd3e79294e4481b46a6421326ac1 [id=99887ef4c8074e5daff46d3f5614cbfe]),no further retry: Not found: Tablet not found: 484375918aed45a9bc93e7ba5ae3a113
I20260320 15:27:21.101094 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd293a68f894cbfa122d53918816c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.101178 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993e2fb1561f4317be85e9b5e5816edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.101513 28734 ts_tablet_manager.cc:1403] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.006s
I20260320 15:27:21.102488 28734 raft_consensus.cc:359] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.106813 28733 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.107098 28733 ts_tablet_manager.cc:1403] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.051s user 0.006s sys 0.001s
I20260320 15:27:21.108104 28733 raft_consensus.cc:359] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.108440 28733 raft_consensus.cc:385] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.108592 28733 raft_consensus.cc:740] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.108945 28733 consensus_queue.cc:260] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.109618 28733 ts_tablet_manager.cc:1434] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:21.109871 28734 raft_consensus.cc:385] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.110222 28564 tablet_service.cc:1511] Processing CreateTablet for tablet b2355eadcae6483bae7ae475d74dd131 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.110543 28733 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:21.110709 28734 raft_consensus.cc:740] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.110263 28425 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3ee3c611d41b483bd35f868042d05 (DEFAULT_TABLE table=default.Table_78dd3a47806c44f186bb967be20e8d9f [id=79ab13b2c3fa42b4b744700487fd8b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.111497 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3ee3c611d41b483bd35f868042d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.112411 28736 tablet_bootstrap.cc:654] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.111843 28734 consensus_queue.cc:260] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.114059 28734 ts_tablet_manager.cc:1434] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260320 15:27:21.114441 28734 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.115852 28048 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 2 got 0
I20260320 15:27:21.116034 28267 tablet_service.cc:1511] Processing CreateTablet for tablet b8f3ee3c611d41b483bd35f868042d05 (DEFAULT_TABLE table=default.Table_78dd3a47806c44f186bb967be20e8d9f [id=79ab13b2c3fa42b4b744700487fd8b8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.116595 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f3ee3c611d41b483bd35f868042d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.110783 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 993e2fb1561f4317be85e9b5e5816edc (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.120502 28045 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 2 got 0
I20260320 15:27:21.121585 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 993e2fb1561f4317be85e9b5e5816edc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.125097 28734 tablet_bootstrap.cc:654] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.128928 28427 tablet_service.cc:1511] Processing CreateTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a (DEFAULT_TABLE table=default.Table_d1cf5412b72c4a58931061b1a1aba1e6 [id=fc92586ccab74ae3960a1eacc8cbac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.129112 28733 tablet_bootstrap.cc:654] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.129525 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcec6175c1ca49f5b682a29d5c1e235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.130025 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2355eadcae6483bae7ae475d74dd131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.132055 28859 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.132280 28859 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.132704 28729 ts_tablet_manager.cc:1929] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.132844 28729 log.cc:1199] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/4902ffdcdbc54d5aa9c71f6cd239c69a
I20260320 15:27:21.133203 28729 ts_tablet_manager.cc:1950] T 4902ffdcdbc54d5aa9c71f6cd239c69a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.133364 28859 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:21.134543 28239 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
W20260320 15:27:21.134778 28240 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:21.134843 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.134898 28240 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.135411 28859 raft_consensus.cc:2749] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.136524 28568 tablet_service.cc:1511] Processing CreateTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 (DEFAULT_TABLE table=default.Table_d2f34f53adf549cc97c5de72f7805087 [id=9f15fd21f99640c7b59422028da7b8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.137060 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bc86d3d1ff4b3aa8463eb21f619917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.139245 28729 tablet_replica.cc:333] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.139480 28729 raft_consensus.cc:2243] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.140317 28729 raft_consensus.cc:2272] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.141175 28734 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.141093 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 093ae84740d5468188b4a56fa16ded9c (DEFAULT_TABLE table=default.Table_20ceb59162bc456f86478ad67ada6956 [id=e79e8aae444f489eb30ffa5b6237e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.141880 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 093ae84740d5468188b4a56fa16ded9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.142163 28729 ts_tablet_manager.cc:1916] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.142140 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet cec531361625474f93daa8453b39f944 (table default.Table_c75f89746474457f81d638cfafc1b8d5 [id=715e4038b9a84544ae18e06efcf9edc9]) successfully deleted
W20260320 15:27:21.142608 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6b6165eab6b0442fa9d94ce577c1b43c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b6165eab6b0442fa9d94ce577c1b43c already in progress: creating tablet
W20260320 15:27:21.142925 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ada1f644b754fbe8b3790b46ec8cdf4 already in progress: creating tablet
I20260320 15:27:21.144917 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 (DEFAULT_TABLE table=default.Table_ce377d5188904a50803bb93b251ac6aa [id=ab1248ac3aab491b8dd0fc75d85cef24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.145396 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591ccc1d349e4fd599d5e343f91defe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.145689 28273 tablet_service.cc:1511] Processing CreateTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a (DEFAULT_TABLE table=default.Table_d1cf5412b72c4a58931061b1a1aba1e6 [id=fc92586ccab74ae3960a1eacc8cbac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.146207 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcec6175c1ca49f5b682a29d5c1e235a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.147924 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 093ae84740d5468188b4a56fa16ded9c (DEFAULT_TABLE table=default.Table_20ceb59162bc456f86478ad67ada6956 [id=e79e8aae444f489eb30ffa5b6237e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.148348 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 093ae84740d5468188b4a56fa16ded9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.149917 28729 ts_tablet_manager.cc:1929] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.150043 28729 log.cc:1199] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1fe4ead926da4389b131d61804451bcf
I20260320 15:27:21.150414 28729 ts_tablet_manager.cc:1950] T 1fe4ead926da4389b131d61804451bcf P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.151633 28729 tablet_replica.cc:333] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.151835 28729 raft_consensus.cc:2243] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.155514 28734 ts_tablet_manager.cc:1403] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.002s
I20260320 15:27:21.157722 28729 raft_consensus.cc:2272] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.158730 28734 raft_consensus.cc:359] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.159018 28734 raft_consensus.cc:385] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.159170 28734 raft_consensus.cc:740] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.158855 28729 ts_tablet_manager.cc:1916] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.159580 28734 consensus_queue.cc:260] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.160264 28734 ts_tablet_manager.cc:1434] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:21.162528 28734 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.164911 28734 tablet_bootstrap.cc:654] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.173588 28734 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.173913 28734 ts_tablet_manager.cc:1403] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260320 15:27:21.174598 28733 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.174823 28733 ts_tablet_manager.cc:1403] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.065s user 0.005s sys 0.000s
I20260320 15:27:21.175086 28734 raft_consensus.cc:359] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.175390 28734 raft_consensus.cc:385] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.175549 28734 raft_consensus.cc:740] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.175846 28733 raft_consensus.cc:359] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.175916 28734 consensus_queue.cc:260] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.176110 28733 raft_consensus.cc:385] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.176239 28733 raft_consensus.cc:740] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.176656 28734 ts_tablet_manager.cc:1434] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:21.176796 28733 consensus_queue.cc:260] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.177106 28734 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.179072 28734 tablet_bootstrap.cc:654] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.182466 28729 ts_tablet_manager.cc:1929] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.185719 28729 log.cc:1199] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/801b3b0ba2344cbe8d89111e17e546ee
I20260320 15:27:21.186388 28735 ts_tablet_manager.cc:1929] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.186650 28729 ts_tablet_manager.cc:1950] T 801b3b0ba2344cbe8d89111e17e546ee P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.186627 28735 log.cc:1199] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/e7e9568395c44af2912efb175fc1704a
I20260320 15:27:21.187841 28733 ts_tablet_manager.cc:1434] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260320 15:27:21.188594 28735 ts_tablet_manager.cc:1950] T e7e9568395c44af2912efb175fc1704a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.189942 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1fe4ead926da4389b131d61804451bcf (table default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]) successfully deleted
I20260320 15:27:21.190387 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet e2b08bcbee8541c998a23dbc1947796a (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
W20260320 15:27:21.190771 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet df5ffaedeaf94def93d93a538c42e3bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df5ffaedeaf94def93d93a538c42e3bd already in progress: creating tablet
W20260320 15:27:21.191138 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 28c1dcc311a14b33a289ae2b8a485d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28c1dcc311a14b33a289ae2b8a485d2b already in progress: creating tablet
I20260320 15:27:21.194840 28730 ts_tablet_manager.cc:1929] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:21.191462 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ea3849a556d94649a3d029b4ac41fcf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea3849a556d94649a3d029b4ac41fcf4 already in progress: creating tablet
I20260320 15:27:21.203157 28045 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
W20260320 15:27:21.204509 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0e8f37a490a4fdfbc2a3dc56f6da297 already in progress: creating tablet
I20260320 15:27:21.205189 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet e7e9568395c44af2912efb175fc1704a (table default.Table_0c2bc058015240478a72fe93a91a9727 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:21.206513 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 2165371f4b5448c7a2625926082ee4c6 (DEFAULT_TABLE table=default.Table_a9fa613ac9d7494a93bb8863f69d4149 [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.207001 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2165371f4b5448c7a2625926082ee4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.207060 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 66a9ddea49e641da881fba339ddef549 (DEFAULT_TABLE table=default.Table_dfa57bec1c394719acd3a8098ed1905d [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.207578 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a9ddea49e641da881fba339ddef549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.207518 28045 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:21.207912 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3bf4fcbc800b49379cb5976acb88f554"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.208498 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bf4fcbc800b49379cb5976acb88f554 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.209962 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd293a68f894cbfa122d53918816c28 (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.210361 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd293a68f894cbfa122d53918816c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.210867 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 47f35263efa3413882954c8c1aa39855 (table default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]) successfully deleted
I20260320 15:27:21.211052 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
W20260320 15:27:21.211261 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6b6165eab6b0442fa9d94ce577c1b43c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b6165eab6b0442fa9d94ce577c1b43c already in progress: creating tablet
W20260320 15:27:21.211441 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ada1f644b754fbe8b3790b46ec8cdf4 already in progress: creating tablet
I20260320 15:27:21.211791 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet dfaf72b4f25c417cafd2f53db42b3315 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
W20260320 15:27:21.212009 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet df5ffaedeaf94def93d93a538c42e3bd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df5ffaedeaf94def93d93a538c42e3bd already in progress: creating tablet
I20260320 15:27:21.212059 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d1cf5412b72c4a58931061b1a1aba1e6" } modify_external_catalogs: true
W20260320 15:27:21.212234 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 28c1dcc311a14b33a289ae2b8a485d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28c1dcc311a14b33a289ae2b8a485d2b already in progress: creating tablet
I20260320 15:27:21.212280 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d1cf5412b72c4a58931061b1a1aba1e6" } modify_external_catalogs: true
I20260320 15:27:21.212358 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 with delete_type TABLET_DATA_DELETED (replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:21.212450 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ea3849a556d94649a3d029b4ac41fcf4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea3849a556d94649a3d029b4ac41fcf4 already in progress: creating tablet
I20260320 15:27:21.212587 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 4902ffdcdbc54d5aa9c71f6cd239c69a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.212646 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (table default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]) successfully deleted
W20260320 15:27:21.212939 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0e8f37a490a4fdfbc2a3dc56f6da297 already in progress: creating tablet
I20260320 15:27:21.212985 28279 tablet_service.cc:1511] Processing CreateTablet for tablet b2355eadcae6483bae7ae475d74dd131 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.213279 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1fe4ead926da4389b131d61804451bcf (table default.Table_e1524b097604437f882f819270d98ea5 [id=aa836d60e1034ab49c285d56163f288a]) successfully deleted
I20260320 15:27:21.213719 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2355eadcae6483bae7ae475d74dd131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.213650 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 (DEFAULT_TABLE table=default.Table_ce377d5188904a50803bb93b251ac6aa [id=ab1248ac3aab491b8dd0fc75d85cef24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.214025 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591ccc1d349e4fd599d5e343f91defe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.195031 28733 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:21.204305 28739 raft_consensus.cc:493] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.214979 28739 raft_consensus.cc:515] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.215422 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.215852 28739 leader_election.cc:290] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:21.216712 28418 tablet_service.cc:1511] Processing CreateTablet for tablet b03e7555b9a646b599acb0c7dff90c9a (DEFAULT_TABLE table=default.Table_a894581517cb4471a20cffa8c3ec26f5 [id=e539cb674eb14059bdb2332a1d1f5bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.217125 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03e7555b9a646b599acb0c7dff90c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.217787 28414 tablet_service.cc:1511] Processing CreateTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 (DEFAULT_TABLE table=default.Table_d2f34f53adf549cc97c5de72f7805087 [id=9f15fd21f99640c7b59422028da7b8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.219370 28417 tablet_service.cc:1511] Processing CreateTablet for tablet b2355eadcae6483bae7ae475d74dd131 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.219772 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bc86d3d1ff4b3aa8463eb21f619917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.220084 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 (DEFAULT_TABLE table=default.Table_ce377d5188904a50803bb93b251ac6aa [id=ab1248ac3aab491b8dd0fc75d85cef24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.220368 28733 tablet_bootstrap.cc:654] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.221019 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2355eadcae6483bae7ae475d74dd131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.221442 28730 log.cc:1199] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:21.221871 28730 ts_tablet_manager.cc:1950] T 0ab709738a3f471da773fd9b523f7fbb P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:21.222553 28730 tablet_replica.cc:333] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.222582 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 66a9ddea49e641da881fba339ddef549 (DEFAULT_TABLE table=default.Table_dfa57bec1c394719acd3a8098ed1905d [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.222754 28730 raft_consensus.cc:2243] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.222947 28730 raft_consensus.cc:2272] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.222975 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a9ddea49e641da881fba339ddef549. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.224381 28270 tablet_service.cc:1511] Processing CreateTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 (DEFAULT_TABLE table=default.Table_d2f34f53adf549cc97c5de72f7805087 [id=9f15fd21f99640c7b59422028da7b8c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.224790 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 9dd293a68f894cbfa122d53918816c28 (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.225572 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 0ab709738a3f471da773fd9b523f7fbb (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
I20260320 15:27:21.226370 28564 tablet_service.cc:1511] Processing CreateTablet for tablet b03e7555b9a646b599acb0c7dff90c9a (DEFAULT_TABLE table=default.Table_a894581517cb4471a20cffa8c3ec26f5 [id=e539cb674eb14059bdb2332a1d1f5bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.226810 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03e7555b9a646b599acb0c7dff90c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.227783 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 (DEFAULT_TABLE table=default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.228324 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a072859f39c4c28ae65b0e9afa79d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.229091 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae4ec6ec99df4fb5b1ebec5b1437c8fd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:21.231271 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 (DEFAULT_TABLE table=default.Table_0a6c79ec44334f829ff10cfe247a257e [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:21.233256 28524 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.236498 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931f6f88a1d94a40b69f07216b8313c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.237139 28729 tablet_replica.cc:333] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.237411 28729 raft_consensus.cc:2243] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.238073 28729 raft_consensus.cc:2272] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.240110 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_104cf9fbc3624b2586dfd717214b89a2" } modify_external_catalogs: true
I20260320 15:27:21.240355 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_104cf9fbc3624b2586dfd717214b89a2" } modify_external_catalogs: true
I20260320 15:27:21.240559 28729 ts_tablet_manager.cc:1916] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.241214 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.241686 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bc3fb47136d4b88a2898af399d74a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.244594 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet bcec6175c1ca49f5b682a29d5c1e235a
I20260320 15:27:21.246294 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.255625 28730 ts_tablet_manager.cc:1916] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.271847 28753 raft_consensus.cc:493] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.272081 28753 raft_consensus.cc:515] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.272644 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b4346c464fb34e9dae8aeccc99dc07a3
I20260320 15:27:21.272818 28753 leader_election.cc:290] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:21.273166 28729 ts_tablet_manager.cc:1929] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.273308 28729 log.cc:1199] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/351a4cc8692a472d9bc5f3dbcc5ee172
I20260320 15:27:21.273389 28730 ts_tablet_manager.cc:1929] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.273525 28730 log.cc:1199] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:21.273731 28729 ts_tablet_manager.cc:1950] T 351a4cc8692a472d9bc5f3dbcc5ee172 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.273921 28730 ts_tablet_manager.cc:1950] T df5ffaedeaf94def93d93a538c42e3bd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:21.274897 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet df5ffaedeaf94def93d93a538c42e3bd (table default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
W20260320 15:27:21.275851 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0c2339ff2c474a288876f32b85cecc72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c2339ff2c474a288876f32b85cecc72 already in progress: creating tablet
I20260320 15:27:21.283847 28568 tablet_service.cc:1511] Processing CreateTablet for tablet c752fb99ef494bda9d144d0e180e3100 (DEFAULT_TABLE table=default.Table_3bf4fcbc800b49379cb5976acb88f554 [id=f330238fc7c246bcb43c472c5f931720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.284482 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c752fb99ef494bda9d144d0e180e3100. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.284652 28382 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8e3284c73cfa4dfba65a88a7dd243190
I20260320 15:27:21.285000 28729 tablet_replica.cc:333] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.285356 28729 raft_consensus.cc:2243] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.285701 28729 raft_consensus.cc:2272] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.286201 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 591ccc1d349e4fd599d5e343f91defe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.286155 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 2165371f4b5448c7a2625926082ee4c6 (DEFAULT_TABLE table=default.Table_a9fa613ac9d7494a93bb8863f69d4149 [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.286916 28729 ts_tablet_manager.cc:1916] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.286996 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae4ec6ec99df4fb5b1ebec5b1437c8fd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:21.288334 28524 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.288493 28524 leader_election.cc:304] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.288949 28739 raft_consensus.cc:2749] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.289371 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 801b3b0ba2344cbe8d89111e17e546ee (table default.Table_04f3937bd89341f98bf5a89d8b00de07 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]) successfully deleted
I20260320 15:27:21.294166 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dd293a68f894cbfa122d53918816c28. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.294802 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 81a1e0957439458083e3a86ac2f01b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81a1e0957439458083e3a86ac2f01b1e already in progress: creating tablet
I20260320 15:27:21.294950 28859 raft_consensus.cc:493] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.295161 28859 raft_consensus.cc:515] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.297223 28729 ts_tablet_manager.cc:1929] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.297498 28729 log.cc:1199] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8e3284c73cfa4dfba65a88a7dd243190
W20260320 15:27:21.297711 28240 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:21.297921 28729 ts_tablet_manager.cc:1950] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.297950 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3284c73cfa4dfba65a88a7dd243190" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:21.298394 28734 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:21.298560 28383 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.298713 28383 leader_election.cc:304] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:21.298925 28383 raft_consensus.cc:2686] T 8e3284c73cfa4dfba65a88a7dd243190 P d22e73471b9046d28d3cf63c905b11b7: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:21.298563 28734 ts_tablet_manager.cc:1403] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.122s user 0.022s sys 0.017s
I20260320 15:27:21.299803 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (table default.Table_66d9e5b794ae4f1298e4264693af9a3c [id=d9929a996d914fb387ec71330be86b22]) successfully deleted
I20260320 15:27:21.300060 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8e3284c73cfa4dfba65a88a7dd243190 (table default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]) successfully deleted
I20260320 15:27:21.301483 28281 tablet_service.cc:1511] Processing CreateTablet for tablet b03e7555b9a646b599acb0c7dff90c9a (DEFAULT_TABLE table=default.Table_a894581517cb4471a20cffa8c3ec26f5 [id=e539cb674eb14059bdb2332a1d1f5bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.301869 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bc86d3d1ff4b3aa8463eb21f619917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.301605 28734 raft_consensus.cc:359] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.302130 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b03e7555b9a646b599acb0c7dff90c9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.302416 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2165371f4b5448c7a2625926082ee4c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.303093 28734 raft_consensus.cc:385] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.303249 28734 raft_consensus.cc:740] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.303606 28734 consensus_queue.cc:260] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.303930 28859 leader_election.cc:290] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:21.308017 28239 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:21.308203 28239 leader_election.cc:304] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.308732 28859 raft_consensus.cc:2749] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.310115 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 (DEFAULT_TABLE table=default.Table_0a6c79ec44334f829ff10cfe247a257e [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.311991 28736 tablet_bootstrap.cc:492] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.311969 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 (DEFAULT_TABLE table=default.Table_0a6c79ec44334f829ff10cfe247a257e [id=19af05ab6c6544b99f6b348f2c52b62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.312310 28736 ts_tablet_manager.cc:1403] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.250s user 0.004s sys 0.001s
I20260320 15:27:21.312377 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931f6f88a1d94a40b69f07216b8313c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.313783 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931f6f88a1d94a40b69f07216b8313c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.315935 28739 raft_consensus.cc:493] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.316118 28739 raft_consensus.cc:515] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.316133 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 (DEFAULT_TABLE table=default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.316572 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a072859f39c4c28ae65b0e9afa79d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.316869 28739 leader_election.cc:290] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:21.318117 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038fd8c34ee48278404acfec3e0b8d8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:21.318889 28524 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.319499 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038fd8c34ee48278404acfec3e0b8d8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:21.320204 28524 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.320351 28524 leader_election.cc:304] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.320834 28739 raft_consensus.cc:2749] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.326674 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 (DEFAULT_TABLE table=default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.327206 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a072859f39c4c28ae65b0e9afa79d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.327356 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.327596 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.327764 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bc3fb47136d4b88a2898af399d74a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.328120 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bc3fb47136d4b88a2898af399d74a8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.333185 28734 ts_tablet_manager.cc:1434] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.033s user 0.000s sys 0.002s
I20260320 15:27:21.337138 28734 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.339342 28734 tablet_bootstrap.cc:654] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:21.342561 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 28c1dcc311a14b33a289ae2b8a485d2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 28c1dcc311a14b33a289ae2b8a485d2b already in progress: creating tablet
I20260320 15:27:21.342917 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.344022 28735 tablet_replica.cc:333] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.344318 28735 raft_consensus.cc:2243] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.344456 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet b4346c464fb34e9dae8aeccc99dc07a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.344612 28735 raft_consensus.cc:2272] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.342083 28730 tablet_replica.cc:333] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.345059 28730 raft_consensus.cc:2243] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.345532 28735 ts_tablet_manager.cc:1916] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.345532 28736 raft_consensus.cc:359] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.347424 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.347625 28736 raft_consensus.cc:385] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.347716 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet b4346c464fb34e9dae8aeccc99dc07a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.347770 28736 raft_consensus.cc:740] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.348183 28736 consensus_queue.cc:260] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.348420 28270 tablet_service.cc:1511] Processing CreateTablet for tablet c752fb99ef494bda9d144d0e180e3100 (DEFAULT_TABLE table=default.Table_3bf4fcbc800b49379cb5976acb88f554 [id=f330238fc7c246bcb43c472c5f931720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.351413 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c752fb99ef494bda9d144d0e180e3100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.348929 28736 ts_tablet_manager.cc:1434] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.036s user 0.000s sys 0.002s
I20260320 15:27:21.355242 28736 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:21.356151 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0c2339ff2c474a288876f32b85cecc72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c2339ff2c474a288876f32b85cecc72 already in progress: creating tablet
I20260320 15:27:21.356673 28427 tablet_service.cc:1511] Processing CreateTablet for tablet c752fb99ef494bda9d144d0e180e3100 (DEFAULT_TABLE table=default.Table_3bf4fcbc800b49379cb5976acb88f554 [id=f330238fc7c246bcb43c472c5f931720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.357118 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c752fb99ef494bda9d144d0e180e3100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.357271 28736 tablet_bootstrap.cc:654] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:21.357234 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 81a1e0957439458083e3a86ac2f01b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81a1e0957439458083e3a86ac2f01b1e already in progress: creating tablet
I20260320 15:27:21.358002 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.358104 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c11ce68814b043f0a072501aeed0ec8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.358716 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c11ce68814b043f0a072501aeed0ec8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.358660 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a5b9cd3e79294e4481b46a6421326ac1" } new_table_name: "default.Table_2f931a99db714d77af814885516e04f7" modify_external_catalogs: true
I20260320 15:27:21.358951 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_469452eb6e4d49b495926119d8ef0568"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.359043 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4df55a5738d24d4292a69c7e5550865b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.359678 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4df55a5738d24d4292a69c7e5550865b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:21.360709 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.359815 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b20f0890d0a046b8b1ec948a07e22f15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.361166 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5a3668da119b4cfea49c6c44f5d7dd54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.361402 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_469452eb6e4d49b495926119d8ef0568 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.362036 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a33f108177064a298842573efef5829c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.347770 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet b4346c464fb34e9dae8aeccc99dc07a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.360883 28733 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.364461 28733 ts_tablet_manager.cc:1403] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.170s user 0.004s sys 0.009s
I20260320 15:27:21.365334 28733 raft_consensus.cc:359] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.365548 28733 raft_consensus.cc:385] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:21.365777 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b20f0890d0a046b8b1ec948a07e22f15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.365621 28733 raft_consensus.cc:740] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.366279 28733 consensus_queue.cc:260] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:21.366606 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a3668da119b4cfea49c6c44f5d7dd54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.367508 28730 raft_consensus.cc:2272] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.369189 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_55bb102a048d48b59fe62e863b719d76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.369812 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55bb102a048d48b59fe62e863b719d76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.370426 28730 ts_tablet_manager.cc:1916] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.371447 28733 ts_tablet_manager.cc:1434] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:21.371866 28733 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:21.374517 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a33f108177064a298842573efef5829c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.374485 28744 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:21.375813 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a9fa613ac9d7494a93bb8863f69d4149" } new_table_name: "default.Table_ba23f221189b481191026b461623223b" modify_external_catalogs: true
I20260320 15:27:21.376088 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.376402 28729 tablet_replica.cc:333] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.376595 28729 raft_consensus.cc:2243] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.376740 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet dfaf72b4f25c417cafd2f53db42b3315 with delete_type TABLET_DATA_DELETED (replaced by tablet 3038fd8c34ee48278404acfec3e0b8d8) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.376803 28729 raft_consensus.cc:2272] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.377125 28744 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:21.377465 28744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:21.377627 28729 ts_tablet_manager.cc:1916] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.379693 28733 tablet_bootstrap.cc:654] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:21.380371 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8f6f683638874e90b0d3c8969c702a5c consensus state has no leader
I20260320 15:27:21.384544 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_638e25af5c024f1e93cf9aff051f145f" } modify_external_catalogs: true
I20260320 15:27:21.386610 28729 ts_tablet_manager.cc:1929] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.386966 28729 log.cc:1199] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:21.387635 28729 ts_tablet_manager.cc:1950] T 9248f0fee2fd446bb8c53891f6e7ba6e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.390065 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_638e25af5c024f1e93cf9aff051f145f" } modify_external_catalogs: true
I20260320 15:27:21.392315 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 1268b25a1c99404aa77e5b1bcc415f48 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:21.398968 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2165371f4b5448c7a2625926082ee4c6 consensus state has no leader
W20260320 15:27:21.400586 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet bcec6175c1ca49f5b682a29d5c1e235a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcec6175c1ca49f5b682a29d5c1e235a already in progress: creating tablet
W20260320 15:27:21.400945 28041 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b4346c464fb34e9dae8aeccc99dc07a3 because the tablet was not found. No further retry: Not found: Tablet not found: b4346c464fb34e9dae8aeccc99dc07a3
I20260320 15:27:21.401185 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet 81a1e0957439458083e3a86ac2f01b1e from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.401371 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9248f0fee2fd446bb8c53891f6e7ba6e (table default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]) successfully deleted
I20260320 15:27:21.401842 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_05e8ba9bd1c24a1694c6b7ef150e9624" } new_table_name: "default.Table_bb5eb65d708a4ece947d11468e68b8d4" modify_external_catalogs: true
W20260320 15:27:21.401911 28041 consensus_peers.cc:597] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 -> Peer ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951): Couldn't send request to peer ee1caebf640d4ec0bf9b28c4f496ca18. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.62:43951: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260320 15:27:21.402094 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_57247cf8e07b450984f71c47c56e7b1b" } modify_external_catalogs: true
I20260320 15:27:21.402277 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_57247cf8e07b450984f71c47c56e7b1b" } modify_external_catalogs: true
I20260320 15:27:21.403458 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet 51ee8b3598384569bbe3d63abb3eaed9 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.405019 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b35323a629d64449b3f893c3049761b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.405651 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b35323a629d64449b3f893c3049761b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.400890 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ce377d5188904a50803bb93b251ac6aa" } modify_external_catalogs: true
I20260320 15:27:21.409549 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ce377d5188904a50803bb93b251ac6aa" } modify_external_catalogs: true
I20260320 15:27:21.412817 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1832ad9fbd5449c9bafcf89adc062c0e" } new_table_name: "default.Table_abe934610fd24a898915bc1d3f4b3307" modify_external_catalogs: true
I20260320 15:27:21.415266 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet 1268b25a1c99404aa77e5b1bcc415f48 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.422650 28733 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.422957 28733 ts_tablet_manager.cc:1403] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.005s
I20260320 15:27:21.424163 28733 raft_consensus.cc:359] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.424520 28733 raft_consensus.cc:385] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.424674 28733 raft_consensus.cc:740] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.425004 28733 consensus_queue.cc:260] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.428524 28733 ts_tablet_manager.cc:1434] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:21.428965 28733 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:21.430151 28734 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.430409 28734 ts_tablet_manager.cc:1403] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.003s
I20260320 15:27:21.431347 28734 raft_consensus.cc:359] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.431596 28733 tablet_bootstrap.cc:654] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.431694 28734 raft_consensus.cc:385] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.431821 28734 raft_consensus.cc:740] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.432030 28730 ts_tablet_manager.cc:1929] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.432301 28730 log.cc:1199] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ea3849a556d94649a3d029b4ac41fcf4
I20260320 15:27:21.432206 28734 consensus_queue.cc:260] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.434403 28734 ts_tablet_manager.cc:1434] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:21.435324 28734 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.437319 28734 tablet_bootstrap.cc:654] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.439675 28730 ts_tablet_manager.cc:1950] T ea3849a556d94649a3d029b4ac41fcf4 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:21.440724 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 203c619513a146f49a4825962f169f45
I20260320 15:27:21.441220 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ea3849a556d94649a3d029b4ac41fcf4 (table default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]) successfully deleted
I20260320 15:27:21.444219 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.444599 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:21.445726 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 203c619513a146f49a4825962f169f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 203c619513a146f49a4825962f169f45 already in progress: creating tablet
W20260320 15:27:21.446717 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0e8f37a490a4fdfbc2a3dc56f6da297 already in progress: creating tablet
W20260320 15:27:21.449019 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d547c032332d4d109222b4743fa28561 consensus state has no leader
I20260320 15:27:21.449457 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 81a1e0957439458083e3a86ac2f01b1e from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.449857 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:21.450387 28046 catalog_manager.cc:6741] ReplaceTablet: tablet 51ee8b3598384569bbe3d63abb3eaed9 deleted and replaced by tablet bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:21.450917 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 with delete_type TABLET_DATA_DELETED (replaced by tablet bb7a910bd2904a40ae63608377206c7e) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.451411 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:21.452419 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 with delete_type TABLET_DATA_DELETED (replaced by tablet bb7a910bd2904a40ae63608377206c7e) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:21.452585 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bb7a910bd2904a40ae63608377206c7e has no consensus state
W20260320 15:27:21.454161 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 51ee8b3598384569bbe3d63abb3eaed9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51ee8b3598384569bbe3d63abb3eaed9 already in progress: creating tablet
I20260320 15:27:21.455193 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.442171 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.440835 28733 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:21.455662 28733 ts_tablet_manager.cc:1403] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260320 15:27:21.456012 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0c2bc058015240478a72fe93a91a9727" } new_table_name: "default.Table_ac4367958ed24d109d328c0c2d1098aa" modify_external_catalogs: true
W20260320 15:27:21.456835 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 591ccc1d349e4fd599d5e343f91defe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591ccc1d349e4fd599d5e343f91defe2 already in progress: creating tablet
I20260320 15:27:21.452736 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.447242 28730 tablet_replica.cc:333] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.458305 28730 raft_consensus.cc:2243] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.458701 28730 raft_consensus.cc:2272] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.459607 28730 ts_tablet_manager.cc:1916] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.460408 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.460836 28729 tablet_replica.cc:333] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.461105 28729 raft_consensus.cc:2243] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.461304 28729 raft_consensus.cc:2272] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.462419 28729 ts_tablet_manager.cc:1916] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.476549 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.478417 28734 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.457166 28733 raft_consensus.cc:359] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.479888 28733 raft_consensus.cc:385] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.480075 28735 ts_tablet_manager.cc:1929] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.480098 28733 raft_consensus.cc:740] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.480489 28734 ts_tablet_manager.cc:1403] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.004s
I20260320 15:27:21.480499 28733 consensus_queue.cc:260] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.480963 28568 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 with delete_type TABLET_DATA_DELETED (replaced by tablet bb7a910bd2904a40ae63608377206c7e) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.481451 28734 raft_consensus.cc:359] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.481768 28735 log.cc:1199] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/4902ffdcdbc54d5aa9c71f6cd239c69a
I20260320 15:27:21.482420 28735 ts_tablet_manager.cc:1950] T 4902ffdcdbc54d5aa9c71f6cd239c69a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.482600 28734 raft_consensus.cc:385] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.482609 28733 ts_tablet_manager.cc:1434] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20260320 15:27:21.482884 28734 raft_consensus.cc:740] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.483448 28734 consensus_queue.cc:260] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.487752 28734 ts_tablet_manager.cc:1434] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:21.489288 28735 tablet_replica.cc:333] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.489964 28734 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.492201 28729 ts_tablet_manager.cc:1929] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.492399 28729 log.cc:1199] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/19c939f8c30943fe8f0619b7dcacc7e0
I20260320 15:27:21.492817 28729 ts_tablet_manager.cc:1950] T 19c939f8c30943fe8f0619b7dcacc7e0 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.493429 28734 tablet_bootstrap.cc:654] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.493639 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 4902ffdcdbc54d5aa9c71f6cd239c69a (table default.Table_53f16cd868254af4833f760b23d22b68 [id=78aabf018bec488ead665868500b9592]) successfully deleted
W20260320 15:27:21.493992 28041 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b4346c464fb34e9dae8aeccc99dc07a3 because the tablet was not found. No further retry: Not found: Tablet not found: b4346c464fb34e9dae8aeccc99dc07a3
W20260320 15:27:21.494220 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bcec6175c1ca49f5b682a29d5c1e235a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcec6175c1ca49f5b682a29d5c1e235a already in progress: creating tablet
I20260320 15:27:21.494976 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.495625 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 19c939f8c30943fe8f0619b7dcacc7e0 (table default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]) successfully deleted
I20260320 15:27:21.500051 28735 raft_consensus.cc:2243] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.500461 28735 raft_consensus.cc:2272] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.501422 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e (DEFAULT_TABLE table=default.Table_5a3668da119b4cfea49c6c44f5d7dd54 [id=dede327e43e74b34b23d244805c6d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.501935 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4b53fb135741efad5d3e59a6d4eb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.502704 28730 ts_tablet_manager.cc:1929] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.502847 28730 log.cc:1199] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:21.503213 28730 ts_tablet_manager.cc:1950] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:21.503865 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9dbd3b0e92a744dcbc132037c67303ab consensus state has no leader
I20260320 15:27:21.500602 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e (DEFAULT_TABLE table=default.Table_5a3668da119b4cfea49c6c44f5d7dd54 [id=dede327e43e74b34b23d244805c6d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.489773 28736 tablet_bootstrap.cc:492] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.504544 28730 tablet_replica.cc:333] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.504709 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4b53fb135741efad5d3e59a6d4eb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.504767 28730 raft_consensus.cc:2243] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.504709 28736 ts_tablet_manager.cc:1403] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.150s user 0.005s sys 0.000s
I20260320 15:27:21.504951 28730 raft_consensus.cc:2272] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.505887 28736 raft_consensus.cc:359] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.506086 28736 raft_consensus.cc:385] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.506194 28736 raft_consensus.cc:740] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.506501 28736 consensus_queue.cc:260] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.507121 28736 ts_tablet_manager.cc:1434] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:21.508208 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (table default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]) successfully deleted
I20260320 15:27:21.508267 28730 ts_tablet_manager.cc:1916] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:21.508512 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0c2339ff2c474a288876f32b85cecc72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c2339ff2c474a288876f32b85cecc72 already in progress: creating tablet
W20260320 15:27:21.508757 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 81a1e0957439458083e3a86ac2f01b1e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81a1e0957439458083e3a86ac2f01b1e already in progress: creating tablet
I20260320 15:27:21.511190 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 47f35263efa3413882954c8c1aa39855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.507498 28736 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.514780 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e (DEFAULT_TABLE table=default.Table_5a3668da119b4cfea49c6c44f5d7dd54 [id=dede327e43e74b34b23d244805c6d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.515416 28735 ts_tablet_manager.cc:1916] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.516112 28733 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:21.516140 28736 tablet_bootstrap.cc:654] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.518345 28739 raft_consensus.cc:493] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.519677 28739 raft_consensus.cc:515] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.521490 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:21.522249 28524 leader_election.cc:343] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.522606 28739 leader_election.cc:290] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:21.523214 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:21.524013 28524 leader_election.cc:343] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.524181 28524 leader_election.cc:304] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.525252 28739 raft_consensus.cc:2749] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.519358 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4b53fb135741efad5d3e59a6d4eb3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.529641 28730 ts_tablet_manager.cc:1929] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.531513 28730 log.cc:1199] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:21.531601 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c57985f062354b93bd560e9d7012fe5e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.532415 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c57985f062354b93bd560e9d7012fe5e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.540776 28427 tablet_service.cc:1511] Processing CreateTablet for tablet edb7b53bb7444a39b471218c6039542f (DEFAULT_TABLE table=default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00 [id=7b5c27e0814148848d52b88c9dce9553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.541445 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7b53bb7444a39b471218c6039542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.543386 28736 tablet_bootstrap.cc:492] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.543641 28736 ts_tablet_manager.cc:1403] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.000s
I20260320 15:27:21.544889 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_dabfce1f58cf47eeb405fae7b61d20c3" } modify_external_catalogs: true
I20260320 15:27:21.545109 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_dabfce1f58cf47eeb405fae7b61d20c3" } modify_external_catalogs: true
I20260320 15:27:21.546651 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c11ce68814b043f0a072501aeed0ec8a" } modify_external_catalogs: true
I20260320 15:27:21.546759 28729 tablet_replica.cc:333] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.546854 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c11ce68814b043f0a072501aeed0ec8a" } modify_external_catalogs: true
I20260320 15:27:21.546996 28729 raft_consensus.cc:2243] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.547225 28729 raft_consensus.cc:2272] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.548923 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 1268b25a1c99404aa77e5b1bcc415f48
I20260320 15:27:21.549503 28050 catalog_manager.cc:6741] ReplaceTablet: tablet 1268b25a1c99404aa77e5b1bcc415f48 deleted and replaced by tablet 89425da569794a8fa972817023bf853d
I20260320 15:27:21.549805 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.550740 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.551138 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:21.551836 28281 tablet_service.cc:1511] Processing CreateTablet for tablet edb7b53bb7444a39b471218c6039542f (DEFAULT_TABLE table=default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00 [id=7b5c27e0814148848d52b88c9dce9553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.552418 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7b53bb7444a39b471218c6039542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.551976 28736 raft_consensus.cc:359] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.552724 28736 raft_consensus.cc:385] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.552821 28736 raft_consensus.cc:740] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.553098 28736 consensus_queue.cc:260] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.555115 28736 ts_tablet_manager.cc:1434] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:21.555680 28736 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.555848 28053 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 2 got 0
I20260320 15:27:21.557622 28736 tablet_bootstrap.cc:654] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.560505 28734 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.560709 28734 ts_tablet_manager.cc:1403] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.006s
I20260320 15:27:21.560967 28053 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 2 got 0
I20260320 15:27:21.561666 28734 raft_consensus.cc:359] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.561896 28734 raft_consensus.cc:385] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.561970 28734 raft_consensus.cc:740] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.562270 28734 consensus_queue.cc:260] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.565279 28053 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]). Expected version 1 got 0
I20260320 15:27:21.565631 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.538295 28735 ts_tablet_manager.cc:1929] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.566076 28735 log.cc:1199] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/dfaf72b4f25c417cafd2f53db42b3315
I20260320 15:27:21.566776 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.567008 28734 ts_tablet_manager.cc:1434] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260320 15:27:21.568310 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b34048a722c3485490637965c60cc5dc
I20260320 15:27:21.568632 28734 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.569559 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 545c8fd2d4c845adba6dbf42dfddba3e
I20260320 15:27:21.570822 28734 tablet_bootstrap.cc:654] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.572175 28046 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_0149a83e663749ef9b7aaf5e635463d5 [id=399646419f3b4fd28458144f43bd4553]). Expected version 1 got 0
I20260320 15:27:21.574828 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.574828 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.575484 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.575830 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.577967 28735 ts_tablet_manager.cc:1950] T dfaf72b4f25c417cafd2f53db42b3315 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.579519 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet dfaf72b4f25c417cafd2f53db42b3315 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
W20260320 15:27:21.580129 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 203c619513a146f49a4825962f169f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 203c619513a146f49a4825962f169f45 already in progress: creating tablet
W20260320 15:27:21.580423 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 51ee8b3598384569bbe3d63abb3eaed9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 51ee8b3598384569bbe3d63abb3eaed9 already in progress: creating tablet
W20260320 15:27:21.580961 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 591ccc1d349e4fd599d5e343f91defe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591ccc1d349e4fd599d5e343f91defe2 already in progress: creating tablet
I20260320 15:27:21.582108 28730 ts_tablet_manager.cc:1950] T 3038fd8c34ee48278404acfec3e0b8d8 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:21.583086 28729 ts_tablet_manager.cc:1916] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.583663 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.584142 28735 tablet_replica.cc:333] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:21.584498 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.585836 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 3038fd8c34ee48278404acfec3e0b8d8 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
W20260320 15:27:21.586087 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet bcec6175c1ca49f5b682a29d5c1e235a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcec6175c1ca49f5b682a29d5c1e235a already in progress: creating tablet
W20260320 15:27:21.586354 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b4346c464fb34e9dae8aeccc99dc07a3 because the tablet was not found. No further retry: Not found: Tablet not found: b4346c464fb34e9dae8aeccc99dc07a3
W20260320 15:27:21.586573 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 203c619513a146f49a4825962f169f45 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 203c619513a146f49a4825962f169f45 already in progress: creating tablet
W20260320 15:27:21.586787 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 591ccc1d349e4fd599d5e343f91defe2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 591ccc1d349e4fd599d5e343f91defe2 already in progress: creating tablet
I20260320 15:27:21.586902 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe (DEFAULT_TABLE table=default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.587445 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.587478 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.587637 28730 tablet_replica.cc:333] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.587778 28734 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.587841 28730 raft_consensus.cc:2243] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.587921 28734 ts_tablet_manager.cc:1403] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.006s
I20260320 15:27:21.588099 28735 raft_consensus.cc:2243] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.588392 28735 raft_consensus.cc:2272] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.588452 28559 tablet_service.cc:1511] Processing CreateTablet for tablet edb7b53bb7444a39b471218c6039542f (DEFAULT_TABLE table=default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00 [id=7b5c27e0814148848d52b88c9dce9553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.588788 28734 raft_consensus.cc:359] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.588975 28734 raft_consensus.cc:385] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.589020 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb7b53bb7444a39b471218c6039542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.589068 28734 raft_consensus.cc:740] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.590001 28730 raft_consensus.cc:2272] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.590741 28730 ts_tablet_manager.cc:1916] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.590896 28734 consensus_queue.cc:260] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.592856 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.595053 28733 tablet_bootstrap.cc:654] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.595041 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.595521 28734 ts_tablet_manager.cc:1434] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:21.595644 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cfad5cf843a437e8e15c7236d1dc5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.599004 28734 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.601600 28734 tablet_bootstrap.cc:654] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.604221 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_953563486c1549d3be5c6add9236d616"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.604878 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_953563486c1549d3be5c6add9236d616 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.605144 28739 raft_consensus.cc:493] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.605341 28739 raft_consensus.cc:515] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.606220 28739 leader_election.cc:290] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:21.607100 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97811e288242436c8e74fd9f45d8d81e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:21.607430 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97811e288242436c8e74fd9f45d8d81e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:21.608403 28524 leader_election.cc:343] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.613659 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d135759507f0411dbdc9e8a3608562d5" } new_table_name: "default.Table_c0173c5015ce4d06aa17f7cbb3ecd511" modify_external_catalogs: true
I20260320 15:27:21.613778 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3787678c407745418136143b1e78fd61"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.614423 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3787678c407745418136143b1e78fd61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.615218 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_99a1f24aff574bff820f96a585a8c417"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.615777 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99a1f24aff574bff820f96a585a8c417 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.615724 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a219d3248ce44f78a7989f8933d4d3fa" } new_table_name: "default.Table_d021f6d697df4823968c7f3b2d5d779c" modify_external_catalogs: true
I20260320 15:27:21.617015 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_28bb9bed388041caa7c69d121e4dbabe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.617563 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28bb9bed388041caa7c69d121e4dbabe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.618650 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2f381f2d934a49e8a9a89d065c7ba856"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.618824 28524 leader_election.cc:343] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.618978 28524 leader_election.cc:304] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:21.619117 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f381f2d934a49e8a9a89d065c7ba856 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.619159 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3a07fada18374bf297e97be27789813f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.619738 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a07fada18374bf297e97be27789813f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.619762 28739 raft_consensus.cc:2749] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.623759 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_38cee1701ad24626b2929b539fdb3c3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.624330 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38cee1701ad24626b2929b539fdb3c3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.627629 28734 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.627835 28734 ts_tablet_manager.cc:1403] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.004s
I20260320 15:27:21.628742 28734 raft_consensus.cc:359] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.628969 28734 raft_consensus.cc:385] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.629261 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet d57566e7907c4783aafce347e46b89e3 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.629558 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe (DEFAULT_TABLE table=default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.630173 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cfad5cf843a437e8e15c7236d1dc5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.629062 28734 raft_consensus.cc:740] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.631474 28734 consensus_queue.cc:260] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.634076 28734 ts_tablet_manager.cc:1434] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260320 15:27:21.634093 28046 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:21.634454 28734 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:21.636415 28422 tablet_service.cc:1511] Processing CreateTablet for tablet b82c47ae3786401d931bb0b486ca3570 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.636900 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82c47ae3786401d931bb0b486ca3570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.640406 28730 ts_tablet_manager.cc:1929] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.640599 28730 log.cc:1199] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:21.640865 28735 ts_tablet_manager.cc:1916] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.641047 28730 ts_tablet_manager.cc:1950] T 51ee8b3598384569bbe3d63abb3eaed9 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:21.641160 28046 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:21.641733 28736 tablet_bootstrap.cc:492] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.642068 28736 ts_tablet_manager.cc:1403] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.005s
I20260320 15:27:21.642086 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 51ee8b3598384569bbe3d63abb3eaed9 (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
W20260320 15:27:21.642616 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 1268b25a1c99404aa77e5b1bcc415f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1268b25a1c99404aa77e5b1bcc415f48 already in progress: creating tablet
I20260320 15:27:21.642998 28730 tablet_replica.cc:333] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:21.643064 28736 raft_consensus.cc:359] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.643210 28730 raft_consensus.cc:2243] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.643280 28736 raft_consensus.cc:385] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.643378 28736 raft_consensus.cc:740] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.643381 28730 raft_consensus.cc:2272] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.644174 28736 consensus_queue.cc:260] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.644709 28736 ts_tablet_manager.cc:1434] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:21.645000 28736 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.646206 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b85eb77d36ad4cd58e04e60d9db9831c" } modify_external_catalogs: true
I20260320 15:27:21.646409 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b85eb77d36ad4cd58e04e60d9db9831c" } modify_external_catalogs: true
I20260320 15:27:21.648206 28270 tablet_service.cc:1511] Processing CreateTablet for tablet b82c47ae3786401d931bb0b486ca3570 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.648634 28736 tablet_bootstrap.cc:654] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.648695 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82c47ae3786401d931bb0b486ca3570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.651034 28734 tablet_bootstrap.cc:654] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:21.655937 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7a8e822249154a1784ebd66f8e56abdf consensus state has no leader
W20260320 15:27:21.658778 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ca38bc990634e21a2c38dd840488f8c consensus state has no leader
I20260320 15:27:21.660589 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet d57566e7907c4783aafce347e46b89e3
I20260320 15:27:21.661173 28051 catalog_manager.cc:6741] ReplaceTablet: tablet d57566e7907c4783aafce347e46b89e3 deleted and replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82
I20260320 15:27:21.663475 28736 tablet_bootstrap.cc:492] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.663712 28736 ts_tablet_manager.cc:1403] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.001s
I20260320 15:27:21.664758 28736 raft_consensus.cc:359] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.665004 28736 raft_consensus.cc:385] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.665098 28736 raft_consensus.cc:740] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.665364 28736 consensus_queue.cc:260] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.666308 28736 ts_tablet_manager.cc:1434] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:21.666668 28736 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.668960 28736 tablet_bootstrap.cc:654] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.674968 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.675637 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0a6c79ec44334f829ff10cfe247a257e" } new_table_name: "default.Table_1f7a90e189124cc49bcbd37f8df5a793" modify_external_catalogs: true
I20260320 15:27:21.678058 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.679714 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_db6589b29a164cbc95f829610dd012d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.680783 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3bf4fcbc800b49379cb5976acb88f554" } new_table_name: "default.Table_be682052d66448438f9d7e7fc47a7ced" modify_external_catalogs: true
I20260320 15:27:21.681303 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5abb1b4446804b0eb71ccecaa7dffaf3" } modify_external_catalogs: true
W20260320 15:27:21.680789 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db6589b29a164cbc95f829610dd012d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.682919 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5abb1b4446804b0eb71ccecaa7dffaf3" } modify_external_catalogs: true
I20260320 15:27:21.688161 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.689496 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.690111 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet 5bc3fb47136d4b88a2898af399d74a8a from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.693181 28730 ts_tablet_manager.cc:1916] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.695156 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 2069aa42660947d8a1d37d6d40846991 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.698442 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e3284c73cfa4dfba65a88a7dd243190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.698477 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_74ed9d4773ad47bf919a3d16d763795a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.699095 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74ed9d4773ad47bf919a3d16d763795a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.702574 28736 tablet_bootstrap.cc:492] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:21.702811 28736 ts_tablet_manager.cc:1403] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.004s
I20260320 15:27:21.703835 28736 raft_consensus.cc:359] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.704115 28736 raft_consensus.cc:385] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.704252 28736 raft_consensus.cc:740] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:21.704981 28736 consensus_queue.cc:260] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.705605 28736 ts_tablet_manager.cc:1434] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:21.706784 28736 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:21.709018 28736 tablet_bootstrap.cc:654] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:21.710099 28416 tablet_service.cc:1511] Processing CreateTablet for tablet b34048a722c3485490637965c60cc5dc (DEFAULT_TABLE table=default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.710493 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 351a4cc8692a472d9bc5f3dbcc5ee172 with delete_type TABLET_DATA_DELETED (replaced by tablet 03d9ae931a354bc2b8e2eb190ca0cff0) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.710651 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34048a722c3485490637965c60cc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.711100 28270 tablet_service.cc:1511] Processing CreateTablet for tablet b34048a722c3485490637965c60cc5dc (DEFAULT_TABLE table=default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.711583 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34048a722c3485490637965c60cc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.715242 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5a3668da119b4cfea49c6c44f5d7dd54" } new_table_name: "default.Table_3796f2b714f64f18a0db9057941513ff" modify_external_catalogs: true
I20260320 15:27:21.708021 28559 tablet_service.cc:1511] Processing CreateTablet for tablet b82c47ae3786401d931bb0b486ca3570 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.718438 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82c47ae3786401d931bb0b486ca3570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.721742 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe (DEFAULT_TABLE table=default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.722306 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cfad5cf843a437e8e15c7236d1dc5fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.728381 28559 tablet_service.cc:1511] Processing CreateTablet for tablet b34048a722c3485490637965c60cc5dc (DEFAULT_TABLE table=default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.729048 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b34048a722c3485490637965c60cc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.729194 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 (DEFAULT_TABLE table=default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.729648 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc27a4968ca4203b67dd3064fb03868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.733173 28561 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a910bd2904a40ae63608377206c7e (DEFAULT_TABLE table=default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.733610 28564 tablet_service.cc:1511] Processing CreateTablet for tablet f70530cd63a0420fb51527759c951e31 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.733747 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a910bd2904a40ae63608377206c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.734015 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70530cd63a0420fb51527759c951e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.739470 28565 tablet_service.cc:1511] Processing CreateTablet for tablet c323b7bcb671464ea6ff584a633f10c0 (DEFAULT_TABLE table=default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.740108 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c323b7bcb671464ea6ff584a633f10c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.741834 28413 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a910bd2904a40ae63608377206c7e (DEFAULT_TABLE table=default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.742435 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a910bd2904a40ae63608377206c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.749117 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 97811e288242436c8e74fd9f45d8d81e
W20260320 15:27:21.750985 28041 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 7a8e822249154a1784ebd66f8e56abdf consensus state has no leader
I20260320 15:27:21.752837 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab (DEFAULT_TABLE table=default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.753500 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa2d48a54dd403c82d7f3a64db5aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.754354 28417 tablet_service.cc:1511] Processing CreateTablet for tablet c323b7bcb671464ea6ff584a633f10c0 (DEFAULT_TABLE table=default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.754505 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab (DEFAULT_TABLE table=default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.755146 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c323b7bcb671464ea6ff584a633f10c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.755350 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa2d48a54dd403c82d7f3a64db5aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.759824 28558 tablet_service.cc:1558] Processing DeleteTablet for tablet 97811e288242436c8e74fd9f45d8d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.764268 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 557d8be0e4bf42239b2bb4585e7d0f06
W20260320 15:27:21.766716 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 931f6f88a1d94a40b69f07216b8313c8 consensus state has no leader
W20260320 15:27:21.767272 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c752fb99ef494bda9d144d0e180e3100 consensus state has no leader
I20260320 15:27:21.767750 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 5bc3fb47136d4b88a2898af399d74a8a
I20260320 15:27:21.768253 28050 catalog_manager.cc:6741] ReplaceTablet: tablet 5bc3fb47136d4b88a2898af399d74a8a deleted and replaced by tablet 35347116811e4aee8142a503dd21843a
W20260320 15:27:21.768949 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7c4b53fb135741efad5d3e59a6d4eb3e consensus state has no leader
I20260320 15:27:21.769446 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 2069aa42660947d8a1d37d6d40846991
I20260320 15:27:21.769963 28053 catalog_manager.cc:6741] ReplaceTablet: tablet 2069aa42660947d8a1d37d6d40846991 deleted and replaced by tablet 66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:21.772611 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.772898 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 2069aa42660947d8a1d37d6d40846991 with delete_type TABLET_DATA_DELETED (replaced by tablet 66a181681cd84b7fb09c3742f4f2222f) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.773093 28558 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.776291 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f40bc6a4b2834fcc860499b3353f1bff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:21.776961 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f40bc6a4b2834fcc860499b3353f1bff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.778661 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b20f0890d0a046b8b1ec948a07e22f15" } modify_external_catalogs: true
I20260320 15:27:21.778880 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b20f0890d0a046b8b1ec948a07e22f15" } modify_external_catalogs: true
I20260320 15:27:21.780484 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_469452eb6e4d49b495926119d8ef0568" } modify_external_catalogs: true
I20260320 15:27:21.780728 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_469452eb6e4d49b495926119d8ef0568" } modify_external_catalogs: true
I20260320 15:27:21.781800 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7da45f21aeef45fbb41fbf00df427cb3" } modify_external_catalogs: true
I20260320 15:27:21.782021 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7da45f21aeef45fbb41fbf00df427cb3" } modify_external_catalogs: true
I20260320 15:27:21.787766 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 97811e288242436c8e74fd9f45d8d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.788619 28416 tablet_service.cc:1511] Processing CreateTablet for tablet f70530cd63a0420fb51527759c951e31 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.793705 28274 tablet_service.cc:1511] Processing CreateTablet for tablet bb7a910bd2904a40ae63608377206c7e (DEFAULT_TABLE table=default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.794226 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb7a910bd2904a40ae63608377206c7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.797233 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.798913 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.809120 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 6d1db9963c4645ae9aab72b4b22d6938 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.810711 28565 tablet_service.cc:1511] Processing CreateTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a (DEFAULT_TABLE table=default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.810952 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:21.811192 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6201b74fdb0428e98f2eebb0b24d01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.813174 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.814795 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70530cd63a0420fb51527759c951e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.818138 28558 tablet_service.cc:1511] Processing CreateTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 (DEFAULT_TABLE table=default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.818624 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f92561764b4cb9b15bb9a310a350f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.822070 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet b82c47ae3786401d931bb0b486ca3570 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:21.822791 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.823243 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet c323b7bcb671464ea6ff584a633f10c0
I20260320 15:27:21.823750 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet da957aed322f46acaf5876c567c2ff94
I20260320 15:27:21.824215 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:21.824687 28053 catalog_manager.cc:6741] ReplaceTablet: tablet 6d1db9963c4645ae9aab72b4b22d6938 deleted and replaced by tablet d56de00174db4e2d8b1c7215038a04aa
I20260320 15:27:21.827415 28566 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.827625 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.829614 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.829682 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.830489 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe (DEFAULT_TABLE table=default.Table_953563486c1549d3be5c6add9236d616 [id=eba1ff6618e0486898840ab0fd2f0882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.830952 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7b964395f24e45bc2715be3fac6dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.835716 28423 tablet_service.cc:1511] Processing CreateTablet for tablet a2775196c8e04dcc80c7e089b57579fd (DEFAULT_TABLE table=default.Table_2f381f2d934a49e8a9a89d065c7ba856 [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.836447 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_76b6b411e37e4bdf807e460a44d52c20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:21.836658 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:21.836617 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 89425da569794a8fa972817023bf853d (DEFAULT_TABLE table=default.Table_35e50a930cbd465b95ca84b3cea68490 [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:21.837023 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76b6b411e37e4bdf807e460a44d52c20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:21.837014 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 2069aa42660947d8a1d37d6d40846991 with delete_type TABLET_DATA_DELETED (replaced by tablet 66a181681cd84b7fb09c3742f4f2222f) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.837200 28052 catalog_manager.cc:6741] ReplaceTablet: tablet b82c47ae3786401d931bb0b486ca3570 deleted and replaced by tablet cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:21.837232 28418 tablet_service.cc:1511] Processing CreateTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a (DEFAULT_TABLE table=default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.837424 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccc693e7704650bd77e0c4ca11862e (DEFAULT_TABLE table=default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.837584 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe (DEFAULT_TABLE table=default.Table_953563486c1549d3be5c6add9236d616 [id=eba1ff6618e0486898840ab0fd2f0882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.837870 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccc693e7704650bd77e0c4ca11862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.838030 28413 tablet_service.cc:1511] Processing CreateTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 (DEFAULT_TABLE table=default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.838516 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.838712 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccc693e7704650bd77e0c4ca11862e (DEFAULT_TABLE table=default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.839021 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2775196c8e04dcc80c7e089b57579fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.842754 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 757b5afdea674217909610e3719a8804 (DEFAULT_TABLE table=default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.843209 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757b5afdea674217909610e3719a8804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.847828 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccc693e7704650bd77e0c4ca11862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.851811 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6201b74fdb0428e98f2eebb0b24d01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.858548 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7b964395f24e45bc2715be3fac6dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.862104 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 757b5afdea674217909610e3719a8804 (DEFAULT_TABLE table=default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.862591 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757b5afdea674217909610e3719a8804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.869923 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 48fa326348104d148abbba907f4c547f (DEFAULT_TABLE table=default.Table_28bb9bed388041caa7c69d121e4dbabe [id=f2e791aa9a37498fa24b1904cfc045ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.873365 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f92561764b4cb9b15bb9a310a350f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.875690 28273 tablet_service.cc:1511] Processing CreateTablet for tablet c323b7bcb671464ea6ff584a633f10c0 (DEFAULT_TABLE table=default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.879017 28735 ts_tablet_manager.cc:1929] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.879184 28735 log.cc:1199] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/47f35263efa3413882954c8c1aa39855
I20260320 15:27:21.879586 28729 ts_tablet_manager.cc:1929] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:21.879833 28729 log.cc:1199] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ea3849a556d94649a3d029b4ac41fcf4
I20260320 15:27:21.880365 28729 ts_tablet_manager.cc:1950] T ea3849a556d94649a3d029b4ac41fcf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:21.880267 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab (DEFAULT_TABLE table=default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.880718 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.880925 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 97811e288242436c8e74fd9f45d8d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.881029 28279 tablet_service.cc:1511] Processing CreateTablet for tablet f70530cd63a0420fb51527759c951e31 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.882754 28729 tablet_replica.cc:333] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:21.883030 28729 raft_consensus.cc:2243] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:21.883292 28729 raft_consensus.cc:2272] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:21.884166 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c323b7bcb671464ea6ff584a633f10c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.884182 28729 ts_tablet_manager.cc:1916] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:21.884665 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89425da569794a8fa972817023bf853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.887646 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 (DEFAULT_TABLE table=default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.888200 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc27a4968ca4203b67dd3064fb03868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.889058 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70530cd63a0420fb51527759c951e31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.891433 28739 raft_consensus.cc:493] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.891652 28739 raft_consensus.cc:515] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.892309 28734 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:21.892381 28739 leader_election.cc:290] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:21.892479 28734 ts_tablet_manager.cc:1403] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.258s user 0.011s sys 0.034s
I20260320 15:27:21.893452 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:21.893483 28734 raft_consensus.cc:359] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.893664 28734 raft_consensus.cc:385] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:21.893837 28734 raft_consensus.cc:740] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:21.894215 28524 leader_election.cc:343] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.894232 28734 consensus_queue.cc:260] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.895548 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fa326348104d148abbba907f4c547f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.896673 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.896955 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.898698 28735 ts_tablet_manager.cc:1950] T 47f35263efa3413882954c8c1aa39855 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:21.898972 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa2d48a54dd403c82d7f3a64db5aeab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.898905 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:21.902495 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ea3849a556d94649a3d029b4ac41fcf4 (table default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]) successfully deleted
I20260320 15:27:21.903295 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 48fa326348104d148abbba907f4c547f (DEFAULT_TABLE table=default.Table_28bb9bed388041caa7c69d121e4dbabe [id=f2e791aa9a37498fa24b1904cfc045ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.903867 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 2069aa42660947d8a1d37d6d40846991 with delete_type TABLET_DATA_DELETED (replaced by tablet 66a181681cd84b7fb09c3742f4f2222f) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.904026 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 (DEFAULT_TABLE table=default.Table_74ed9d4773ad47bf919a3d16d763795a [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.904157 28268 tablet_service.cc:1511] Processing CreateTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a (DEFAULT_TABLE table=default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.904721 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6201b74fdb0428e98f2eebb0b24d01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.904675 28414 tablet_service.cc:1511] Processing CreateTablet for tablet cc97a5f2bd964a05bb7b2427226debdd (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.905675 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 5b263f9a5df2477e8dc4cd52e5834991 (DEFAULT_TABLE table=default.Table_db6589b29a164cbc95f829610dd012d0 [id=3120af75e43943d68a8cd8ef413acb4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.906517 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 8a18f32c29224bd883ceb20658dff228 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.907122 28426 tablet_service.cc:1511] Processing CreateTablet for tablet d56de00174db4e2d8b1c7215038a04aa (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.907291 28271 tablet_service.cc:1511] Processing CreateTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 (DEFAULT_TABLE table=default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.907971 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5f92561764b4cb9b15bb9a310a350f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.908054 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 35347116811e4aee8142a503dd21843a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.908172 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 5a20d7f22cfa403184e73b24d565b60a (DEFAULT_TABLE table=default.Table_c57985f062354b93bd560e9d7012fe5e [id=a010951cfbe240fea932452776c43ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.908684 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.909125 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.909389 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.909667 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.910476 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d7e4d26994a17bf2d2615d6552412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.914794 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 (DEFAULT_TABLE table=default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.915405 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc27a4968ca4203b67dd3064fb03868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.915848 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc97a5f2bd964a05bb7b2427226debdd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.917084 28524 leader_election.cc:343] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.917253 28524 leader_election.cc:304] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:21.918352 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a18f32c29224bd883ceb20658dff228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.919705 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 89425da569794a8fa972817023bf853d (DEFAULT_TABLE table=default.Table_35e50a930cbd465b95ca84b3cea68490 [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.923249 28739 raft_consensus.cc:2749] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:21.923776 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56de00174db4e2d8b1c7215038a04aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.924573 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 5a20d7f22cfa403184e73b24d565b60a (DEFAULT_TABLE table=default.Table_c57985f062354b93bd560e9d7012fe5e [id=a010951cfbe240fea932452776c43ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.925940 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe (DEFAULT_TABLE table=default.Table_953563486c1549d3be5c6add9236d616 [id=eba1ff6618e0486898840ab0fd2f0882]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.926350 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.927906 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35347116811e4aee8142a503dd21843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.928826 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 (DEFAULT_TABLE table=default.Table_74ed9d4773ad47bf919a3d16d763795a [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.930137 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.931368 28859 raft_consensus.cc:493] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.931746 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b263f9a5df2477e8dc4cd52e5834991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.931613 28859 raft_consensus.cc:515] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.932554 28859 leader_election.cc:290] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:21.934003 28552 tablet_service.cc:1511] Processing CreateTablet for tablet cc97a5f2bd964a05bb7b2427226debdd (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.934600 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:21.935644 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 5b263f9a5df2477e8dc4cd52e5834991 (DEFAULT_TABLE table=default.Table_db6589b29a164cbc95f829610dd012d0 [id=3120af75e43943d68a8cd8ef413acb4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.936151 28549 tablet_service.cc:1511] Processing CreateTablet for tablet d56de00174db4e2d8b1c7215038a04aa (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.936373 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 89425da569794a8fa972817023bf853d (DEFAULT_TABLE table=default.Table_35e50a930cbd465b95ca84b3cea68490 [id=c4badbffe34944afa17c48384084a51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.937340 28272 tablet_service.cc:1511] Processing CreateTablet for tablet a2775196c8e04dcc80c7e089b57579fd (DEFAULT_TABLE table=default.Table_2f381f2d934a49e8a9a89d065c7ba856 [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.937098 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 8a18f32c29224bd883ceb20658dff228 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.937601 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 35347116811e4aee8142a503dd21843a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.938189 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 757b5afdea674217909610e3719a8804 (DEFAULT_TABLE table=default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.939376 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 48fa326348104d148abbba907f4c547f (DEFAULT_TABLE table=default.Table_28bb9bed388041caa7c69d121e4dbabe [id=f2e791aa9a37498fa24b1904cfc045ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.940541 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 27ccc693e7704650bd77e0c4ca11862e (DEFAULT_TABLE table=default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.942456 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 5a20d7f22cfa403184e73b24d565b60a (DEFAULT_TABLE table=default.Table_c57985f062354b93bd560e9d7012fe5e [id=a010951cfbe240fea932452776c43ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.944622 28269 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:21.946972 28269 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (DEFAULT_TABLE table=default.Table_50e316065f1e43459edfe03e6c2468b7 [id=b7fdfea8538d4e758051c53a61442e69]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:21.949174 28239 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8e3284c73cfa4dfba65a88a7dd243190
I20260320 15:27:21.949400 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 66a181681cd84b7fb09c3742f4f2222f (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.951388 28562 tablet_service.cc:1511] Processing CreateTablet for tablet a2775196c8e04dcc80c7e089b57579fd (DEFAULT_TABLE table=default.Table_2f381f2d934a49e8a9a89d065c7ba856 [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.951884 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 (DEFAULT_TABLE table=default.Table_74ed9d4773ad47bf919a3d16d763795a [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.954879 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fa326348104d148abbba907f4c547f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.960022 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a20d7f22cfa403184e73b24d565b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.964708 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6f8c20b5904eafb9a62d14bb155c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.969724 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:21.970584 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 66a181681cd84b7fb09c3742f4f2222f (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:21.971256 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a181681cd84b7fb09c3742f4f2222f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.902787 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 1268b25a1c99404aa77e5b1bcc415f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1268b25a1c99404aa77e5b1bcc415f48 already in progress: creating tablet
W20260320 15:27:21.965096 28040 catalog_manager.cc:5075] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): alter failed for tablet 484375918aed45a9bc93e7ba5ae3a113 (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]),no further retry: Not found: Tablet not found: 484375918aed45a9bc93e7ba5ae3a113
W20260320 15:27:21.976528 28041 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 73693.6us
W20260320 15:27:21.976989 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 14be8e6ed89b413a84f9517ca5ff02dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14be8e6ed89b413a84f9517ca5ff02dc already in progress: creating tablet
I20260320 15:27:21.977793 28753 raft_consensus.cc:493] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:21.977977 28753 raft_consensus.cc:515] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.978756 28753 leader_election.cc:290] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:21.980165 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:21.980298 28053 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
W20260320 15:27:21.980947 28383 leader_election.cc:343] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.980981 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:21.981213 28588 raft_consensus.cc:2468] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:21.981732 28382 leader_election.cc:304] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:21.982692 28753 raft_consensus.cc:2804] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:21.982826 28753 raft_consensus.cc:493] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:21.982918 28753 raft_consensus.cc:3060] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:21.983174 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e7b964395f24e45bc2715be3fac6dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.984927 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d7e4d26994a17bf2d2615d6552412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.986217 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2775196c8e04dcc80c7e089b57579fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.987049 28753 raft_consensus.cc:515] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:21.988479 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:21.988893 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48fa326348104d148abbba907f4c547f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:21.989356 28383 leader_election.cc:343] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:21.990072 28753 leader_election.cc:290] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:21.990394 28269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6f8c20b5904eafb9a62d14bb155c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.992504 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a20d7f22cfa403184e73b24d565b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.992865 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89425da569794a8fa972817023bf853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.995594 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a20d7f22cfa403184e73b24d565b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.998332 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 757b5afdea674217909610e3719a8804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:21.999576 28734 ts_tablet_manager.cc:1434] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260320 15:27:22.001663 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ccc693e7704650bd77e0c4ca11862e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.005143 28734 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:22.007941 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 545c8fd2d4c845adba6dbf42dfddba3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 545c8fd2d4c845adba6dbf42dfddba3e already in progress: creating tablet
W20260320 15:27:22.008464 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b34048a722c3485490637965c60cc5dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b34048a722c3485490637965c60cc5dc already in progress: creating tablet
I20260320 15:27:22.010026 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2f381f2d934a49e8a9a89d065c7ba856" } new_table_name: "default.Table_58424e8752fe4921bafb617ee7eb20de" modify_external_catalogs: true
I20260320 15:27:22.010105 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f9e940b8c37f46a6a671a47344d1c74d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.010233 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0149a83e663749ef9b7aaf5e635463d5" } new_table_name: "default.Table_248df89af9b1443e8ef145d0cf33ae62" modify_external_catalogs: true
I20260320 15:27:22.010296 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4165a4481d3840ceb77849e30d57927d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.011583 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2b163b340195441daf46b7d424a4ebdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.011644 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.012382 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.012475 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a894581517cb4471a20cffa8c3ec26f5" } modify_external_catalogs: true
I20260320 15:27:22.012508 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5d5d276ee59e4bc68cecddd3cff48146"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.012663 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a894581517cb4471a20cffa8c3ec26f5" } modify_external_catalogs: true
I20260320 15:27:22.014636 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b263f9a5df2477e8dc4cd52e5834991. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.017940 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:22.017998 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4165a4481d3840ceb77849e30d57927d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.018307 28053 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
W20260320 15:27:22.019054 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9e940b8c37f46a6a671a47344d1c74d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:22.019276 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d5d276ee59e4bc68cecddd3cff48146 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:22.024417 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:22.025810 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b163b340195441daf46b7d424a4ebdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.029165 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6f8c20b5904eafb9a62d14bb155c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.029536 28734 tablet_bootstrap.cc:654] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.032509 28736 tablet_bootstrap.cc:492] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.032802 28736 ts_tablet_manager.cc:1403] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.326s user 0.008s sys 0.011s
I20260320 15:27:22.033859 28729 ts_tablet_manager.cc:1929] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.033819 28736 raft_consensus.cc:359] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.034209 28736 raft_consensus.cc:385] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.034341 28736 raft_consensus.cc:740] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.034785 28736 consensus_queue.cc:260] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.036343 28736 ts_tablet_manager.cc:1434] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:22.036768 28736 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.036700 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_618b12e348124dfab7d293a70b297a41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.037432 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_618b12e348124dfab7d293a70b297a41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.040629 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2d7e4d26994a17bf2d2615d6552412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.041320 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56de00174db4e2d8b1c7215038a04aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.041829 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc97a5f2bd964a05bb7b2427226debdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.042001 28729 log.cc:1199] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:22.042438 28729 ts_tablet_manager.cc:1950] T 0ab709738a3f471da773fd9b523f7fbb P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:22.045356 28736 tablet_bootstrap.cc:654] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.092536 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87a3d5e4ff8f42409482af49c636be0c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:22.092885 28588 raft_consensus.cc:3060] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.093509 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35347116811e4aee8142a503dd21843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.098491 28739 raft_consensus.cc:493] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.098773 28739 raft_consensus.cc:515] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.099785 28739 leader_election.cc:290] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:22.103561 28283 tablet_service.cc:1511] Processing CreateTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b (DEFAULT_TABLE table=default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.104179 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb65832eefe4bd4b08d95aa83c7bd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.105289 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a18f32c29224bd883ceb20658dff228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.111591 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a181681cd84b7fb09c3742f4f2222f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.112310 28735 tablet_replica.cc:333] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.112668 28735 raft_consensus.cc:2243] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.112912 28735 raft_consensus.cc:2272] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.114076 28735 ts_tablet_manager.cc:1916] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.117367 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89425da569794a8fa972817023bf853d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.121125 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 35347116811e4aee8142a503dd21843a (DEFAULT_TABLE table=default.Table_ee7fbcc0eb0e4f7e961909167e813202 [id=695e9628a1d947dd981b9f954f3f3072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.121870 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35347116811e4aee8142a503dd21843a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.126124 28859 raft_consensus.cc:493] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.126366 28859 raft_consensus.cc:515] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.127152 28859 leader_election.cc:290] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.127835 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2775196c8e04dcc80c7e089b57579fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.128554 28239 leader_election.cc:343] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 351a4cc8692a472d9bc5f3dbcc5ee172
I20260320 15:27:22.134115 28274 tablet_service.cc:1511] Processing CreateTablet for tablet cc97a5f2bd964a05bb7b2427226debdd (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.134820 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc97a5f2bd964a05bb7b2427226debdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.139989 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 8a18f32c29224bd883ceb20658dff228 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.140815 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a18f32c29224bd883ceb20658dff228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.141268 28739 raft_consensus.cc:493] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.141484 28739 raft_consensus.cc:515] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.142328 28739 leader_election.cc:290] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:22.148995 28753 raft_consensus.cc:493] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.149221 28753 raft_consensus.cc:515] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.150136 28753 leader_election.cc:290] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.151301 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5ffaedeaf94def93d93a538c42e3bd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.151859 28279 tablet_service.cc:1511] Processing CreateTablet for tablet d56de00174db4e2d8b1c7215038a04aa (DEFAULT_TABLE table=default.Table_501982d031484c3eb2382cf055a9d3be [id=cc5e59bc215e444ba6c156b6d0ded84c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.152483 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d56de00174db4e2d8b1c7215038a04aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.152549 28383 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: Tablet not RUNNING: STOPPED
W20260320 15:27:22.152742 28382 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:22.152863 28382 leader_election.cc:304] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:22.153479 28753 raft_consensus.cc:2749] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.153878 28564 tablet_service.cc:1511] Processing CreateTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b (DEFAULT_TABLE table=default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:22.164481 28240 leader_election.cc:343] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 8e3284c73cfa4dfba65a88a7dd243190
I20260320 15:27:22.164722 28240 leader_election.cc:304] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.165002 28859 raft_consensus.cc:2749] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:22.166090 28240 leader_election.cc:343] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 351a4cc8692a472d9bc5f3dbcc5ee172
I20260320 15:27:22.166239 28240 leader_election.cc:304] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.166488 28859 raft_consensus.cc:2749] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:22.167698 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d21b0e8db1a424ea329b8f9cd03ab5b consensus state has no leader
I20260320 15:27:22.168143 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_69c8a46cb085490e8a0e7b689e10fa36" } modify_external_catalogs: true
I20260320 15:27:22.168349 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_69c8a46cb085490e8a0e7b689e10fa36" } modify_external_catalogs: true
I20260320 15:27:22.168574 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_35e50a930cbd465b95ca84b3cea68490" } new_table_name: "default.Table_9f87c6321cdc405ab18f525e320707f2" modify_external_catalogs: true
I20260320 15:27:22.168805 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ee7fbcc0eb0e4f7e961909167e813202" } new_table_name: "default.Table_37e1fe279d384188986393c967dcb9de" modify_external_catalogs: true
I20260320 15:27:22.168929 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 66a181681cd84b7fb09c3742f4f2222f (DEFAULT_TABLE table=default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.169409 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a181681cd84b7fb09c3742f4f2222f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.170647 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:22.171044 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_dfa57bec1c394719acd3a8098ed1905d" } new_table_name: "default.Table_f6315615237b426ca7b2aea87662ecc9" modify_external_catalogs: true
I20260320 15:27:22.171423 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_eb27eb94efef46238ae0dd4cb3f3bfac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.171993 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb27eb94efef46238ae0dd4cb3f3bfac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.172899 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a4dec0096d0946b280892bd8fd4ee5a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.173415 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bce978bcf8414d928d5f77eaf4e90cbe" } modify_external_catalogs: true
W20260320 15:27:22.173522 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4dec0096d0946b280892bd8fd4ee5a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.173611 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bce978bcf8414d928d5f77eaf4e90cbe" } modify_external_catalogs: true
I20260320 15:27:22.174706 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_abe934610fd24a898915bc1d3f4b3307" } modify_external_catalogs: true
I20260320 15:27:22.175037 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_abe934610fd24a898915bc1d3f4b3307" } modify_external_catalogs: true
I20260320 15:27:22.175154 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d2f34f53adf549cc97c5de72f7805087" } modify_external_catalogs: true
I20260320 15:27:22.175333 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d2f34f53adf549cc97c5de72f7805087" } modify_external_catalogs: true
W20260320 15:27:22.175827 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2775196c8e04dcc80c7e089b57579fd consensus state has no leader
I20260320 15:27:22.177476 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 026bb078f3904a44820442e69e654f72 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:22.184088 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 5b263f9a5df2477e8dc4cd52e5834991 (DEFAULT_TABLE table=default.Table_db6589b29a164cbc95f829610dd012d0 [id=3120af75e43943d68a8cd8ef413acb4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.184895 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b263f9a5df2477e8dc4cd52e5834991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.192866 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet da860b779a174adbb4fd6f7f7520348c
I20260320 15:27:22.193650 28049 master_service.cc:954] ReplaceTablet: received request to replace tablet da957aed322f46acaf5876c567c2ff94 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:22.195276 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 66a9ddea49e641da881fba339ddef549 consensus state has no leader
W20260320 15:27:22.195832 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35347116811e4aee8142a503dd21843a consensus state has no leader
W20260320 15:27:22.196273 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 89425da569794a8fa972817023bf853d consensus state has no leader
I20260320 15:27:22.201332 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.201370 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2069aa42660947d8a1d37d6d40846991" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.191769 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:22.202682 28524 leader_election.cc:343] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.204857 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb65832eefe4bd4b08d95aa83c7bd7b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.205004 28524 leader_election.cc:343] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:22.208709 28733 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.568s user 0.007s sys 0.011s
W20260320 15:27:22.208823 28733 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.569s user 0.007s sys 0.011s
I20260320 15:27:22.208878 28736 tablet_bootstrap.cc:492] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.209172 28736 ts_tablet_manager.cc:1403] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.172s user 0.008s sys 0.040s
I20260320 15:27:22.210294 28733 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:22.210223 28736 raft_consensus.cc:359] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.210438 28736 raft_consensus.cc:385] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.210687 28733 ts_tablet_manager.cc:1403] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.695s user 0.011s sys 0.012s
I20260320 15:27:22.211516 28736 raft_consensus.cc:740] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.212353 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.212400 28736 consensus_queue.cc:260] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.214406 28753 raft_consensus.cc:493] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.214601 28753 raft_consensus.cc:515] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.214776 28736 ts_tablet_manager.cc:1434] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:22.215116 28736 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.215361 28753 leader_election.cc:290] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.216495 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.216773 28587 raft_consensus.cc:2468] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:22.217702 28382 leader_election.cc:304] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:22.217722 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.217934 28300 raft_consensus.cc:2468] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:22.218228 28753 raft_consensus.cc:493] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.218406 28753 raft_consensus.cc:515] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.219310 28753 leader_election.cc:290] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.219506 28753 raft_consensus.cc:2804] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:22.219798 28753 raft_consensus.cc:493] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:22.219904 28753 raft_consensus.cc:3060] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.220361 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbd3b0e92a744dcbc132037c67303ab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:22.221983 28383 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.222623 28753 raft_consensus.cc:515] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.223978 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:22.224212 28300 raft_consensus.cc:3060] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.225992 28735 ts_tablet_manager.cc:1929] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.226184 28735 log.cc:1199] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:22.226783 28735 ts_tablet_manager.cc:1950] T df5ffaedeaf94def93d93a538c42e3bd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.227604 28300 raft_consensus.cc:2468] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:22.228410 28383 leader_election.cc:304] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:22.229175 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.230082 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 801b3b0ba2344cbe8d89111e17e546ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.230877 28917 raft_consensus.cc:2804] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.231479 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbd3b0e92a744dcbc132037c67303ab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.231813 28587 raft_consensus.cc:2468] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:22.232286 28382 leader_election.cc:304] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:22.233011 28753 leader_election.cc:290] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.233613 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.233937 28753 raft_consensus.cc:2804] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:22.235033 28753 raft_consensus.cc:493] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:22.235271 28753 raft_consensus.cc:3060] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.237958 28734 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.238227 28734 ts_tablet_manager.cc:1403] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.233s user 0.056s sys 0.028s
I20260320 15:27:22.239353 28734 raft_consensus.cc:359] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.239758 28734 raft_consensus.cc:385] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.239936 28734 raft_consensus.cc:740] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.240311 28734 consensus_queue.cc:260] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.241222 28735 tablet_replica.cc:333] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.241416 28735 raft_consensus.cc:2243] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.241609 28735 raft_consensus.cc:2272] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.242287 28735 ts_tablet_manager.cc:1916] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.242871 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0ab709738a3f471da773fd9b523f7fbb (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
W20260320 15:27:22.243288 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d57566e7907c4783aafce347e46b89e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d57566e7907c4783aafce347e46b89e3 already in progress: creating tablet
W20260320 15:27:22.243643 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 97811e288242436c8e74fd9f45d8d81e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97811e288242436c8e74fd9f45d8d81e already in progress: creating tablet
I20260320 15:27:22.243824 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet ea3849a556d94649a3d029b4ac41fcf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:22.244067 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 5bc3fb47136d4b88a2898af399d74a8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bc3fb47136d4b88a2898af399d74a8a already in progress: creating tablet
W20260320 15:27:22.244352 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 557d8be0e4bf42239b2bb4585e7d0f06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 557d8be0e4bf42239b2bb4585e7d0f06 already in progress: creating tablet
W20260320 15:27:22.244611 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9aa2d48a54dd403c82d7f3a64db5aeab already in progress: creating tablet
W20260320 15:27:22.244865 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6d1db9963c4645ae9aab72b4b22d6938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d1db9963c4645ae9aab72b4b22d6938 already in progress: creating tablet
W20260320 15:27:22.245117 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2069aa42660947d8a1d37d6d40846991 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2069aa42660947d8a1d37d6d40846991 already in progress: creating tablet
W20260320 15:27:22.245383 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet da957aed322f46acaf5876c567c2ff94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da957aed322f46acaf5876c567c2ff94 already in progress: creating tablet
W20260320 15:27:22.245623 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b82c47ae3786401d931bb0b486ca3570 already in progress: creating tablet
I20260320 15:27:22.257812 28736 tablet_bootstrap.cc:654] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.258755 28735 ts_tablet_manager.cc:1929] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.258878 28735 log.cc:1199] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8e3284c73cfa4dfba65a88a7dd243190
I20260320 15:27:22.259238 28735 ts_tablet_manager.cc:1950] T 8e3284c73cfa4dfba65a88a7dd243190 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.260346 28734 ts_tablet_manager.cc:1434] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260320 15:27:22.246806 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.247329 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.247644 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.247840 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2069aa42660947d8a1d37d6d40846991" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.241832 28416 tablet_service.cc:1511] Processing CreateTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b (DEFAULT_TABLE table=default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.262876 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb65832eefe4bd4b08d95aa83c7bd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.265604 28735 tablet_replica.cc:333] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.265931 28735 raft_consensus.cc:2243] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.266153 28735 raft_consensus.cc:2272] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.266225 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 (DEFAULT_TABLE table=default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.266739 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f207dee674a4288c8c60b5d3ff9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.266914 28735 ts_tablet_manager.cc:1916] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.267503 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba738a76a424c18b36985a1977bcee4 (DEFAULT_TABLE table=default.Table_4165a4481d3840ceb77849e30d57927d [id=56aecaca2e3a4c868f419fa30e404f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.238354 28753 raft_consensus.cc:515] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.260766 28917 raft_consensus.cc:697] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:22.270421 28736 tablet_bootstrap.cc:492] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.270654 28736 ts_tablet_manager.cc:1403] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.008s
I20260320 15:27:22.270797 28917 consensus_queue.cc:237] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.271142 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbd3b0e92a744dcbc132037c67303ab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:22.271579 28736 raft_consensus.cc:359] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:22.271759 28383 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.271833 28736 raft_consensus.cc:385] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.271903 28736 raft_consensus.cc:740] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.272151 28736 consensus_queue.cc:260] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.274024 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:22.274904 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:22.275421 28736 ts_tablet_manager.cc:1434] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260320 15:27:22.276332 28734 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.279148 28734 tablet_bootstrap.cc:654] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.279781 28736 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.281982 28736 tablet_bootstrap.cc:654] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:22.282924 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c323b7bcb671464ea6ff584a633f10c0 already in progress: creating tablet
I20260320 15:27:22.283617 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 47f35263efa3413882954c8c1aa39855 (table default.Table_c61291fb416b42aeb522d95afd859670 [id=96a227e5c2734c579a86db52e8380b24]) successfully deleted
W20260320 15:27:22.285179 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1268b25a1c99404aa77e5b1bcc415f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1268b25a1c99404aa77e5b1bcc415f48 already in progress: creating tablet
I20260320 15:27:22.285518 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b1bc86d3d1ff4b3aa8463eb21f619917
I20260320 15:27:22.284909 28735 ts_tablet_manager.cc:1929] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.283485 28753 leader_election.cc:290] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:22.285554 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 14be8e6ed89b413a84f9517ca5ff02dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14be8e6ed89b413a84f9517ca5ff02dc already in progress: creating tablet
W20260320 15:27:22.294771 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 545c8fd2d4c845adba6dbf42dfddba3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 545c8fd2d4c845adba6dbf42dfddba3e already in progress: creating tablet
I20260320 15:27:22.295617 28735 log.cc:1199] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/351a4cc8692a472d9bc5f3dbcc5ee172
W20260320 15:27:22.295631 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b34048a722c3485490637965c60cc5dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b34048a722c3485490637965c60cc5dc already in progress: creating tablet
W20260320 15:27:22.295905 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d57566e7907c4783aafce347e46b89e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d57566e7907c4783aafce347e46b89e3 already in progress: creating tablet
I20260320 15:27:22.296072 28735 ts_tablet_manager.cc:1950] T 351a4cc8692a472d9bc5f3dbcc5ee172 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.297062 28734 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.299573 28735 tablet_replica.cc:333] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.299613 28736 tablet_bootstrap.cc:492] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.299777 28736 ts_tablet_manager.cc:1403] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.002s
I20260320 15:27:22.299810 28735 raft_consensus.cc:2243] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.299863 28734 ts_tablet_manager.cc:1403] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260320 15:27:22.300068 28735 raft_consensus.cc:2272] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.300987 28736 raft_consensus.cc:359] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.301281 28736 raft_consensus.cc:385] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.301222 28734 raft_consensus.cc:359] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.301429 28736 raft_consensus.cc:740] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.301488 28735 ts_tablet_manager.cc:1916] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.301863 28736 consensus_queue.cc:260] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.305400 28736 ts_tablet_manager.cc:1434] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:22.306254 28734 raft_consensus.cc:385] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.306720 28734 raft_consensus.cc:740] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.308745 28734 consensus_queue.cc:260] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.309253 28735 ts_tablet_manager.cc:1929] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.309391 28735 log.cc:1199] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/801b3b0ba2344cbe8d89111e17e546ee
I20260320 15:27:22.309417 28734 ts_tablet_manager.cc:1434] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260320 15:27:22.309790 28735 ts_tablet_manager.cc:1950] T 801b3b0ba2344cbe8d89111e17e546ee P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.310284 28736 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.312808 28736 tablet_bootstrap.cc:654] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.315956 28735 tablet_replica.cc:333] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.316208 28735 raft_consensus.cc:2243] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.316426 28735 raft_consensus.cc:2272] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.319154 28736 tablet_bootstrap.cc:492] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.319566 28735 ts_tablet_manager.cc:1916] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.331082 28735 ts_tablet_manager.cc:1929] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.331278 28735 log.cc:1199] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ea3849a556d94649a3d029b4ac41fcf4
I20260320 15:27:22.309789 28734 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.331993 28736 ts_tablet_manager.cc:1403] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260320 15:27:22.332883 28736 raft_consensus.cc:359] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.333099 28736 raft_consensus.cc:385] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.333189 28736 raft_consensus.cc:740] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.333590 28736 consensus_queue.cc:260] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.334911 28736 ts_tablet_manager.cc:1434] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:22.335947 28736 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.337378 28735 ts_tablet_manager.cc:1950] T ea3849a556d94649a3d029b4ac41fcf4 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.338796 28736 tablet_bootstrap.cc:654] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.339968 28588 raft_consensus.cc:2468] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:22.339991 28524 leader_election.cc:343] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.340178 28524 leader_election.cc:304] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:22.340521 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 55572.6us
W20260320 15:27:22.340941 28524 leader_election.cc:343] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.341086 28524 leader_election.cc:304] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.341473 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.342496 28734 tablet_bootstrap.cc:654] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.343847 28730 ts_tablet_manager.cc:1929] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.344033 28730 log.cc:1199] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:22.341517 28739 raft_consensus.cc:2749] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.350526 28736 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.350904 28736 ts_tablet_manager.cc:1403] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260320 15:27:22.351720 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.352434 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 7394e941c99e4358ad7a33db84b15348 (DEFAULT_TABLE table=default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.352442 28736 raft_consensus.cc:359] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.352711 28736 raft_consensus.cc:385] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.352816 28736 raft_consensus.cc:740] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.353091 28739 raft_consensus.cc:2749] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.353186 28736 consensus_queue.cc:260] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.354617 28736 ts_tablet_manager.cc:1434] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:22.354899 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba738a76a424c18b36985a1977bcee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.355706 28736 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.356419 28739 raft_consensus.cc:493] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.356660 28739 raft_consensus.cc:515] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.358001 28736 tablet_bootstrap.cc:654] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.361135 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0242a7968541f480367b45cda5d9cc (DEFAULT_TABLE table=default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.361873 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 (DEFAULT_TABLE table=default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.362926 28739 leader_election.cc:290] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:22.364524 28920 raft_consensus.cc:493] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.364802 28920 raft_consensus.cc:515] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.366529 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.366732 28736 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.366962 28736 ts_tablet_manager.cc:1403] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260320 15:27:22.367607 28549 tablet_service.cc:1511] Processing CreateTablet for tablet db81df6d3e9d49efb94c4f24de9db429 (DEFAULT_TABLE table=default.Table_618b12e348124dfab7d293a70b297a41 [id=d272c148809a44bea0d673f063e2d677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.368319 28736 raft_consensus.cc:359] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.368754 28736 raft_consensus.cc:385] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.368906 28736 raft_consensus.cc:740] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:22.369073 28383 leader_election.cc:343] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.369361 28736 consensus_queue.cc:260] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.370887 28736 ts_tablet_manager.cc:1434] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
W20260320 15:27:22.309931 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 77653.3us
I20260320 15:27:22.371673 28736 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.371989 28920 leader_election.cc:290] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.372766 28551 tablet_service.cc:1511] Processing CreateTablet for tablet e13f1a65661949258f521dc847e674cd (DEFAULT_TABLE table=default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c [id=7dc7c07f0ba646fbb6e8ac8b228e6e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.374060 28050 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:22.374310 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7394e941c99e4358ad7a33db84b15348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.374642 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5438fdc6e6944e79ac8aecd0ef774467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.374900 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db81df6d3e9d49efb94c4f24de9db429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.375340 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f1a65661949258f521dc847e674cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.376075 28733 raft_consensus.cc:359] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.376386 28733 raft_consensus.cc:385] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.376564 28733 raft_consensus.cc:740] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.376744 28736 tablet_bootstrap.cc:654] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.376921 28733 consensus_queue.cc:260] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.379060 28733 ts_tablet_manager.cc:1434] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.168s user 0.002s sys 0.000s
I20260320 15:27:22.380159 28050 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:22.381258 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.381577 28730 ts_tablet_manager.cc:1950] T 14be8e6ed89b413a84f9517ca5ff02dc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:22.385219 28736 tablet_bootstrap.cc:492] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.385563 28736 ts_tablet_manager.cc:1403] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260320 15:27:22.386848 28736 raft_consensus.cc:359] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.387094 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 14be8e6ed89b413a84f9517ca5ff02dc (table default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]) successfully deleted
I20260320 15:27:22.387217 28736 raft_consensus.cc:385] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.387396 28736 raft_consensus.cc:740] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.387802 28736 consensus_queue.cc:260] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.388527 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0242a7968541f480367b45cda5d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.389393 28736 ts_tablet_manager.cc:1434] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:22.389832 28736 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:22.390995 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 545c8fd2d4c845adba6dbf42dfddba3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 545c8fd2d4c845adba6dbf42dfddba3e already in progress: creating tablet
I20260320 15:27:22.392650 28736 tablet_bootstrap.cc:654] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.392745 28045 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
W20260320 15:27:22.396170 28040 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b34048a722c3485490637965c60cc5dc because the tablet was not found. No further retry: Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
I20260320 15:27:22.394140 28733 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:22.401516 28736 tablet_bootstrap.cc:492] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.401860 28736 ts_tablet_manager.cc:1403] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260320 15:27:22.402894 28736 raft_consensus.cc:359] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.403332 28736 raft_consensus.cc:385] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.403338 28050 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
I20260320 15:27:22.403473 28736 raft_consensus.cc:740] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.403859 28736 consensus_queue.cc:260] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.405251 28736 ts_tablet_manager.cc:1434] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260320 15:27:22.405493 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 64546.7us
I20260320 15:27:22.406407 28736 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.407083 28045 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_248df89af9b1443e8ef145d0cf33ae62 [id=399646419f3b4fd28458144f43bd4553]). Expected version 2 got 0
I20260320 15:27:22.407881 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993c9b6ae14ae98c2239abd65333ea" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.408270 28859 raft_consensus.cc:493] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.408437 28859 raft_consensus.cc:515] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.408612 28730 tablet_replica.cc:333] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:22.408823 28730 raft_consensus.cc:2243] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.409174 28859 leader_election.cc:290] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.409544 28736 tablet_bootstrap.cc:654] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.409874 28730 raft_consensus.cc:2272] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.410061 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3038fd8c34ee48278404acfec3e0b8d8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.410552 28730 ts_tablet_manager.cc:1916] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:22.410991 28240 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.416246 28045 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_248df89af9b1443e8ef145d0cf33ae62 [id=399646419f3b4fd28458144f43bd4553]). Expected version 2 got 0
W20260320 15:27:22.416956 28239 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:22.417147 28239 leader_election.cc:304] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.417721 28859 raft_consensus.cc:2749] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.418651 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993c9b6ae14ae98c2239abd65333ea" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.419059 28736 tablet_bootstrap.cc:492] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.419296 28736 ts_tablet_manager.cc:1403] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260320 15:27:22.420392 28736 raft_consensus.cc:359] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:22.420572 28041 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 48377.5us
I20260320 15:27:22.420804 28736 raft_consensus.cc:385] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.421005 28736 raft_consensus.cc:740] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.421142 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet df5ffaedeaf94def93d93a538c42e3bd (table default.Table_a719aa4da12a4d159826d615a6a2302a [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
I20260320 15:27:22.421440 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8e3284c73cfa4dfba65a88a7dd243190 (table default.Table_3c2ca98bf22746c3b0fb188339710bfa [id=b812344ad6934f77bde6f0c31f259940]) successfully deleted
I20260320 15:27:22.421380 28736 consensus_queue.cc:260] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.421674 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 351a4cc8692a472d9bc5f3dbcc5ee172 (table default.Table_66d9e5b794ae4f1298e4264693af9a3c [id=d9929a996d914fb387ec71330be86b22]) successfully deleted
W20260320 15:27:22.421926 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 557d8be0e4bf42239b2bb4585e7d0f06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 557d8be0e4bf42239b2bb4585e7d0f06 already in progress: creating tablet
W20260320 15:27:22.422150 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 97811e288242436c8e74fd9f45d8d81e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 97811e288242436c8e74fd9f45d8d81e already in progress: creating tablet
W20260320 15:27:22.422338 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet da957aed322f46acaf5876c567c2ff94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da957aed322f46acaf5876c567c2ff94 already in progress: creating tablet
W20260320 15:27:22.422537 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6d1db9963c4645ae9aab72b4b22d6938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d1db9963c4645ae9aab72b4b22d6938 already in progress: creating tablet
I20260320 15:27:22.422827 28736 ts_tablet_manager.cc:1434] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:22.424451 28736 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.427416 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "026bb078f3904a44820442e69e654f72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:22.428242 28736 tablet_bootstrap.cc:654] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.432775 28382 leader_election.cc:304] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:22.433620 28920 raft_consensus.cc:2804] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.433835 28920 raft_consensus.cc:697] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:22.434101 28920 consensus_queue.cc:237] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.434383 28588 raft_consensus.cc:3060] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.437644 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 442ca186c9d4449dade39271dec5c608 (DEFAULT_TABLE table=default.Table_eb27eb94efef46238ae0dd4cb3f3bfac [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.438709 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dbd3b0e92a744dcbc132037c67303ab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:22.441793 28524 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.444675 28734 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.444905 28734 ts_tablet_manager.cc:1403] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.135s user 0.002s sys 0.002s
I20260320 15:27:22.445629 28734 raft_consensus.cc:359] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.445847 28734 raft_consensus.cc:385] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.445951 28734 raft_consensus.cc:740] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.446256 28734 consensus_queue.cc:260] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.446787 28734 ts_tablet_manager.cc:1434] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:22.447095 28734 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.448580 28734 tablet_bootstrap.cc:654] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.451108 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.451478 28586 raft_consensus.cc:2468] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:22.422729 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2069aa42660947d8a1d37d6d40846991 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2069aa42660947d8a1d37d6d40846991 already in progress: creating tablet
W20260320 15:27:22.452143 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9aa2d48a54dd403c82d7f3a64db5aeab already in progress: creating tablet
W20260320 15:27:22.452386 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c323b7bcb671464ea6ff584a633f10c0 already in progress: creating tablet
I20260320 15:27:22.453053 28382 leader_election.cc:304] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:22.453354 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (DEFAULT_TABLE table=default.Table_a4dec0096d0946b280892bd8fd4ee5a4 [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.453814 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6231fe30c5b34d0bb7a233abbffeb1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.455106 28920 raft_consensus.cc:2804] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260320 15:27:22.452589 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 5bc3fb47136d4b88a2898af399d74a8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bc3fb47136d4b88a2898af399d74a8a already in progress: creating tablet
I20260320 15:27:22.455214 28920 raft_consensus.cc:493] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:22.455318 28920 raft_consensus.cc:3060] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:22.455462 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b82c47ae3786401d931bb0b486ca3570 already in progress: creating tablet
W20260320 15:27:22.455729 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet da860b779a174adbb4fd6f7f7520348c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da860b779a174adbb4fd6f7f7520348c already in progress: creating tablet
I20260320 15:27:22.455905 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 801b3b0ba2344cbe8d89111e17e546ee (table default.Table_04f3937bd89341f98bf5a89d8b00de07 [id=cfc9769e1acf40bcbf2fd73912a1fc9a]) successfully deleted
W20260320 15:27:22.456100 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b03e7555b9a646b599acb0c7dff90c9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b03e7555b9a646b599acb0c7dff90c9a already in progress: creating tablet
I20260320 15:27:22.456259 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ea3849a556d94649a3d029b4ac41fcf4 (table default.Table_c4feb7323b9748c9b7f01574beecc884 [id=b303fe35a2ff485aa49f17a5b569b35f]) successfully deleted
I20260320 15:27:22.456995 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d3fccc4dd32348d1addde0067aeae899"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.457877 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3fccc4dd32348d1addde0067aeae899 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.458333 28920 raft_consensus.cc:515] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.459456 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0703f70973474e499d7916aa2a63f195"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.460063 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0703f70973474e499d7916aa2a63f195 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.461256 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4fa9df7189e84297bb2b7b04ae79052f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.461433 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b7b75bdac4ca4c17b7971a6fe6da8c00" } new_table_name: "default.Table_1a8f81dd2a33405b819cad145fe64901" modify_external_catalogs: true
W20260320 15:27:22.461907 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fa9df7189e84297bb2b7b04ae79052f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.462898 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:22.463137 28300 raft_consensus.cc:3060] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.463814 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2b163b340195441daf46b7d424a4ebdf" } modify_external_catalogs: true
I20260320 15:27:22.464016 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_2b163b340195441daf46b7d424a4ebdf" } modify_external_catalogs: true
I20260320 15:27:22.465382 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b91605000af04532ba5d3fc895d8dc1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.465989 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3618306c0eff4c74935789df59e679f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.466125 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b91605000af04532ba5d3fc895d8dc1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.466358 28734 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:22.466490 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3618306c0eff4c74935789df59e679f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.466521 28734 ts_tablet_manager.cc:1403] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20260320 15:27:22.467464 28734 raft_consensus.cc:359] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.467705 28734 raft_consensus.cc:385] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.467803 28734 raft_consensus.cc:740] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.467919 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d046770ad1554426ab6791919207cdcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.468490 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d046770ad1554426ab6791919207cdcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.469704 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba738a76a424c18b36985a1977bcee4 (DEFAULT_TABLE table=default.Table_4165a4481d3840ceb77849e30d57927d [id=56aecaca2e3a4c868f419fa30e404f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.470257 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba738a76a424c18b36985a1977bcee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.471599 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 026bb078f3904a44820442e69e654f72
I20260320 15:27:22.472236 28052 catalog_manager.cc:6741] ReplaceTablet: tablet 026bb078f3904a44820442e69e654f72 deleted and replaced by tablet ca5e0c215f574b8aafec8d5e2da9a399
I20260320 15:27:22.472711 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c19d1d44b172435dbb61e970e1b3ced2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.473270 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c19d1d44b172435dbb61e970e1b3ced2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.473945 28300 raft_consensus.cc:2468] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:22.474606 28383 leader_election.cc:304] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:22.474969 28917 raft_consensus.cc:2804] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.475167 28917 raft_consensus.cc:697] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:22.475433 28917 consensus_queue.cc:237] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.476122 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_613a6835f9f44c3392f000337fb9dfdd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:22.476902 28920 leader_election.cc:290] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:22.476915 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_613a6835f9f44c3392f000337fb9dfdd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.478693 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d21b0e8db1a424ea329b8f9cd03ab5b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:22.478891 28586 raft_consensus.cc:3060] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.479588 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bb078f3904a44820442e69e654f72 with delete_type TABLET_DATA_DELETED (replaced by tablet ca5e0c215f574b8aafec8d5e2da9a399) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.480674 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 6ba738a76a424c18b36985a1977bcee4 (DEFAULT_TABLE table=default.Table_4165a4481d3840ceb77849e30d57927d [id=56aecaca2e3a4c868f419fa30e404f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.481189 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ba738a76a424c18b36985a1977bcee4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.488663 28524 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.490485 28587 raft_consensus.cc:3060] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.492394 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_eb27eb94efef46238ae0dd4cb3f3bfac" } new_table_name: "default.Table_e4276cc8c36a438b8f87af916b9fe8e3" modify_external_catalogs: true
I20260320 15:27:22.495256 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e4cb7c7492b24f5caedd6760d05e6190"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.496111 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4cb7c7492b24f5caedd6760d05e6190 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.496037 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 (DEFAULT_TABLE table=default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.497905 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f207dee674a4288c8c60b5d3ff9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.498692 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_76b6b411e37e4bdf807e460a44d52c20" } modify_external_catalogs: true
I20260320 15:27:22.500100 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 (DEFAULT_TABLE table=default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.502075 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.499043 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_76b6b411e37e4bdf807e460a44d52c20" } modify_external_catalogs: true
I20260320 15:27:22.503968 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.506806 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442ca186c9d4449dade39271dec5c608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.505322 28734 consensus_queue.cc:260] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.506121 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906f207dee674a4288c8c60b5d3ff9f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.510421 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7394e941c99e4358ad7a33db84b15348
I20260320 15:27:22.509666 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 with delete_type TABLET_DATA_DELETED (replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.511520 28735 tablet_replica.cc:333] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.511498 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_97a6eb082b53476a8140573e2c89dfca" } new_table_name: "default.Table_d27df02667144e339d2b08cacb4b5e36" modify_external_catalogs: true
I20260320 15:27:22.511765 28735 raft_consensus.cc:2243] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.511994 28735 raft_consensus.cc:2272] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.512706 28735 ts_tablet_manager.cc:1916] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.513455 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.515424 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f5cf94af44a94088b01b125cc76ec506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:22.516085 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5cf94af44a94088b01b125cc76ec506 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:22.516185 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_953563486c1549d3be5c6add9236d616" } new_table_name: "default.Table_c783052b0eb045ea95d77f3722e1ca8b" modify_external_catalogs: true
I20260320 15:27:22.520066 28734 ts_tablet_manager.cc:1434] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.053s user 0.003s sys 0.000s
I20260320 15:27:22.520435 28734 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.521263 28735 ts_tablet_manager.cc:1929] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.521405 28735 log.cc:1199] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0ee4aafbcc4e46bb9dbb0460a8575282
I20260320 15:27:22.521862 28735 ts_tablet_manager.cc:1950] T 0ee4aafbcc4e46bb9dbb0460a8575282 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.522389 28734 tablet_bootstrap.cc:654] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.523836 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.528371 28736 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.528676 28736 ts_tablet_manager.cc:1403] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.105s user 0.029s sys 0.052s
I20260320 15:27:22.529323 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.529688 28736 raft_consensus.cc:359] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.529901 28736 raft_consensus.cc:385] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.529984 28736 raft_consensus.cc:740] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.530280 28736 consensus_queue.cc:260] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.531039 28524 leader_election.cc:304] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.532177 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.533207 28736 ts_tablet_manager.cc:1434] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:22.534309 28736 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.535333 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ab709738a3f471da773fd9b523f7fbb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.535624 28735 tablet_replica.cc:333] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.535840 28735 raft_consensus.cc:2243] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.536023 28735 raft_consensus.cc:2272] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:22.537426 28524 raft_consensus.cc:2686] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:22.538316 28735 ts_tablet_manager.cc:1916] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.539301 28425 tablet_service.cc:1511] Processing CreateTablet for tablet db81df6d3e9d49efb94c4f24de9db429 (DEFAULT_TABLE table=default.Table_618b12e348124dfab7d293a70b297a41 [id=d272c148809a44bea0d673f063e2d677]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:22.539618 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 93996.6us
I20260320 15:27:22.540524 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.541970 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db81df6d3e9d49efb94c4f24de9db429. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.543171 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet edb7b53bb7444a39b471218c6039542f consensus state has no leader
I20260320 15:27:22.544023 28736 tablet_bootstrap.cc:654] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.545185 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a4dec0096d0946b280892bd8fd4ee5a4" } new_table_name: "default.Table_80ba3729fee5441b983da0a2323b6ecb" modify_external_catalogs: true
I20260320 15:27:22.546836 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet f640a6ae307140d2aee3992333e96d84 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:22.547276 28735 ts_tablet_manager.cc:1929] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.547418 28735 log.cc:1199] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0ab709738a3f471da773fd9b523f7fbb
I20260320 15:27:22.547876 28735 ts_tablet_manager.cc:1950] T 0ab709738a3f471da773fd9b523f7fbb P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.550793 28733 tablet_bootstrap.cc:654] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.550858 28284 tablet_service.cc:1511] Processing CreateTablet for tablet db81df6d3e9d49efb94c4f24de9db429 (DEFAULT_TABLE table=default.Table_618b12e348124dfab7d293a70b297a41 [id=d272c148809a44bea0d673f063e2d677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.551340 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db81df6d3e9d49efb94c4f24de9db429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.556334 28736 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.556596 28736 ts_tablet_manager.cc:1403] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.003s
I20260320 15:27:22.557802 28736 raft_consensus.cc:359] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.558073 28736 raft_consensus.cc:385] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.558208 28736 raft_consensus.cc:740] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.558621 28736 consensus_queue.cc:260] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.562989 28734 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.563216 28734 ts_tablet_manager.cc:1403] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.001s
I20260320 15:27:22.564132 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0242a7968541f480367b45cda5d9cc (DEFAULT_TABLE table=default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.564158 28734 raft_consensus.cc:359] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.564471 28734 raft_consensus.cc:385] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.564565 28734 raft_consensus.cc:740] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.564920 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0242a7968541f480367b45cda5d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.564873 28734 consensus_queue.cc:260] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.567273 28734 ts_tablet_manager.cc:1434] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:22.568221 28734 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:22.569870 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 442ca186c9d4449dade39271dec5c608 consensus state has no leader
I20260320 15:27:22.571007 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:22.571811 28734 tablet_bootstrap.cc:654] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:22.572136 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f640a6ae307140d2aee3992333e96d84 consensus state has no leader
I20260320 15:27:22.581049 28734 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.581378 28734 ts_tablet_manager.cc:1403] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260320 15:27:22.582463 28734 raft_consensus.cc:359] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.582792 28734 raft_consensus.cc:385] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.582953 28734 raft_consensus.cc:740] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.583730 28734 consensus_queue.cc:260] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.584615 28734 ts_tablet_manager.cc:1434] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:22.585289 28734 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.586231 28588 raft_consensus.cc:2468] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:22.587960 28734 tablet_bootstrap.cc:654] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.590165 28736 ts_tablet_manager.cc:1434] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260320 15:27:22.590658 28736 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.590580 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 7f0242a7968541f480367b45cda5d9cc (DEFAULT_TABLE table=default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.591043 28859 raft_consensus.cc:493] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.591228 28859 raft_consensus.cc:515] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.593977 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.598557 28734 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.598915 28734 ts_tablet_manager.cc:1403] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.009s
I20260320 15:27:22.600204 28734 raft_consensus.cc:359] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.600631 28734 raft_consensus.cc:385] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.600780 28734 raft_consensus.cc:740] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.601428 28734 consensus_queue.cc:260] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.601728 28587 raft_consensus.cc:2468] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:22.602346 28734 ts_tablet_manager.cc:1434] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:22.602921 28734 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.606051 28734 tablet_bootstrap.cc:654] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.612082 28382 leader_election.cc:304] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:22.613117 28917 raft_consensus.cc:2804] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.613322 28917 raft_consensus.cc:697] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:22.613624 28917 consensus_queue.cc:237] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.616166 28586 raft_consensus.cc:2468] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:22.616928 28734 tablet_bootstrap.cc:492] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.617137 28734 ts_tablet_manager.cc:1403] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260320 15:27:22.618628 28734 raft_consensus.cc:359] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.618942 28734 raft_consensus.cc:385] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.619071 28734 raft_consensus.cc:740] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.619398 28734 consensus_queue.cc:260] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.620595 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f0242a7968541f480367b45cda5d9cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.620795 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet f640a6ae307140d2aee3992333e96d84
I20260320 15:27:22.621306 28052 catalog_manager.cc:6741] ReplaceTablet: tablet f640a6ae307140d2aee3992333e96d84 deleted and replaced by tablet 7feb8477ebf841a19977a434f365f537
W20260320 15:27:22.623049 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6231fe30c5b34d0bb7a233abbffeb1e8 consensus state has no leader
I20260320 15:27:22.620252 28734 ts_tablet_manager.cc:1434] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:22.622068 28753 raft_consensus.cc:493] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.623883 28753 raft_consensus.cc:515] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.624640 28753 leader_election.cc:290] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.624650 28734 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.625444 28859 leader_election.cc:290] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:22.626154 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3e7b964395f24e45bc2715be3fac6dbe consensus state has no leader
I20260320 15:27:22.626276 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ee8b3598384569bbe3d63abb3eaed9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.626888 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.621433 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 7394e941c99e4358ad7a33db84b15348 (DEFAULT_TABLE table=default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.627959 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7394e941c99e4358ad7a33db84b15348. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:22.628044 28240 leader_election.cc:343] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.628571 28734 tablet_bootstrap.cc:654] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:22.628690 28383 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.629340 28418 tablet_service.cc:1511] Processing CreateTablet for tablet e13f1a65661949258f521dc847e674cd (DEFAULT_TABLE table=default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c [id=7dc7c07f0ba646fbb6e8ac8b228e6e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.629796 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f1a65661949258f521dc847e674cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.638139 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 (DEFAULT_TABLE table=default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.638764 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5438fdc6e6944e79ac8aecd0ef774467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.639250 28272 tablet_service.cc:1511] Processing CreateTablet for tablet e13f1a65661949258f521dc847e674cd (DEFAULT_TABLE table=default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c [id=7dc7c07f0ba646fbb6e8ac8b228e6e5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.639910 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e13f1a65661949258f521dc847e674cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.641386 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 7394e941c99e4358ad7a33db84b15348 (DEFAULT_TABLE table=default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.641954 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7394e941c99e4358ad7a33db84b15348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.643285 28048 catalog_manager.cc:5671] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:22.643857 28048 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:22.644158 28048 catalog_manager.cc:5671] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:22.644593 28048 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_248df89af9b1443e8ef145d0cf33ae62 [id=399646419f3b4fd28458144f43bd4553]). Expected version 2 got 0
I20260320 15:27:22.650599 28549 tablet_service.cc:1511] Processing CreateTablet for tablet ca5e0c215f574b8aafec8d5e2da9a399 (DEFAULT_TABLE table=default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.655697 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e0c215f574b8aafec8d5e2da9a399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.659690 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet f640a6ae307140d2aee3992333e96d84 with delete_type TABLET_DATA_DELETED (replaced by tablet 7feb8477ebf841a19977a434f365f537) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:22.660307 28382 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:22.660593 28382 leader_election.cc:304] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:22.660929 28753 raft_consensus.cc:2749] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.663854 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.664113 28587 raft_consensus.cc:2468] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:22.664804 28239 leader_election.cc:304] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.665310 28859 raft_consensus.cc:2804] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:22.665437 28859 raft_consensus.cc:493] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:22.665521 28859 raft_consensus.cc:3060] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.668040 28753 raft_consensus.cc:493] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.668233 28753 raft_consensus.cc:515] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.668937 28753 leader_election.cc:290] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.670011 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae4ec6ec99df4fb5b1ebec5b1437c8fd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:22.669471 28859 raft_consensus.cc:515] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.671372 28859 leader_election.cc:290] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.671355 28048 catalog_manager.cc:5671] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:22.671830 28048 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
I20260320 15:27:22.672447 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:22.672657 28587 raft_consensus.cc:3060] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:22.672798 28383 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.673058 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:22.674758 28240 leader_election.cc:343] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.683663 28859 raft_consensus.cc:493] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.684015 28859 raft_consensus.cc:515] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.685654 28859 leader_election.cc:290] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.685633 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395977eb0444bcf8563cd8774fab40f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.685881 28447 raft_consensus.cc:2468] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:22.686120 28859 raft_consensus.cc:493] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.686365 28859 raft_consensus.cc:515] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:22.686462 28382 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:22.686617 28382 leader_election.cc:304] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:22.686821 28240 leader_election.cc:304] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:22.686945 28917 raft_consensus.cc:2749] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.691558 28859 leader_election.cc:290] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.691850 28859 raft_consensus.cc:2804] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:22.691960 28859 raft_consensus.cc:493] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:22.692101 28859 raft_consensus.cc:3060] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.692190 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395977eb0444bcf8563cd8774fab40f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.692444 28588 raft_consensus.cc:2468] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
W20260320 15:27:22.694677 28239 leader_election.cc:343] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:22.696467 28730 ts_tablet_manager.cc:1929] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.696624 28730 log.cc:1199] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:22.697793 28730 ts_tablet_manager.cc:1950] T 9b993c9b6ae14ae98c2239abd65333ea P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:22.698709 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 147) took 1106 ms. Trace:
I20260320 15:27:22.700273 28736 tablet_bootstrap.cc:654] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.699801 28730 rpcz_store.cc:276] 0320 15:27:21.591903 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.592362 (+ 459us) service_pool.cc:225] Handling call
0320 15:27:22.395492 (+803130us) ts_tablet_manager.cc:1151] Deleting tablet 9b993c9b6ae14ae98c2239abd65333ea
0320 15:27:22.395508 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:22.585958 (+190450us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:22.695795 (+109837us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:22.698656 (+ 2861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":802475}
I20260320 15:27:22.700736 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9b993c9b6ae14ae98c2239abd65333ea (table default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]) successfully deleted
I20260320 15:27:22.701009 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 442ca186c9d4449dade39271dec5c608 (DEFAULT_TABLE table=default.Table_eb27eb94efef46238ae0dd4cb3f3bfac [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.701669 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442ca186c9d4449dade39271dec5c608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.709323 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 149) took 1042 ms. Trace:
I20260320 15:27:22.710040 28587 raft_consensus.cc:2468] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:22.710707 28239 leader_election.cc:304] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.710947 28736 tablet_bootstrap.cc:492] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.711171 28736 ts_tablet_manager.cc:1403] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.121s user 0.002s sys 0.002s
I20260320 15:27:22.712769 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (DEFAULT_TABLE table=default.Table_a4dec0096d0946b280892bd8fd4ee5a4 [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:22.712986 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b03e7555b9a646b599acb0c7dff90c9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b03e7555b9a646b599acb0c7dff90c9a already in progress: creating tablet
W20260320 15:27:22.713245 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet da860b779a174adbb4fd6f7f7520348c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da860b779a174adbb4fd6f7f7520348c already in progress: creating tablet
I20260320 15:27:22.713611 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6231fe30c5b34d0bb7a233abbffeb1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.713454 28859 raft_consensus.cc:515] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:22.713681 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7a910bd2904a40ae63608377206c7e already in progress: creating tablet
W20260320 15:27:22.713886 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b395977eb0444bcf8563cd8774fab40f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b395977eb0444bcf8563cd8774fab40f already in progress: creating tablet
W20260320 15:27:22.714128 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b1bc86d3d1ff4b3aa8463eb21f619917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1bc86d3d1ff4b3aa8463eb21f619917 already in progress: creating tablet
I20260320 15:27:22.715406 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bb078f3904a44820442e69e654f72 with delete_type TABLET_DATA_DELETED (replaced by tablet ca5e0c215f574b8aafec8d5e2da9a399) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.715930 28729 tablet_replica.cc:333] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:22.716217 28729 raft_consensus.cc:2243] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:22.716403 28736 raft_consensus.cc:359] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.716571 28729 pending_rounds.cc:70] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:22.716670 28736 raft_consensus.cc:385] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.716650 28729 pending_rounds.cc:77] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387731547611136 op_type: NO_OP noop_request { }
I20260320 15:27:22.716750 28736 raft_consensus.cc:740] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.716794 28729 raft_consensus.cc:2889] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:22.716905 28729 raft_consensus.cc:2272] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.717054 28736 consensus_queue.cc:260] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.717839 28859 leader_election.cc:290] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.717967 28729 ts_tablet_manager.cc:1916] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.718436 28859 raft_consensus.cc:2804] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.718619 28859 raft_consensus.cc:697] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:22.719089 28859 consensus_queue.cc:237] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.721529 28736 ts_tablet_manager.cc:1434] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260320 15:27:22.723716 28734 tablet_bootstrap.cc:492] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.723927 28734 ts_tablet_manager.cc:1403] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.099s user 0.011s sys 0.033s
I20260320 15:27:22.725543 28734 raft_consensus.cc:359] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.725795 28734 raft_consensus.cc:385] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.725883 28734 raft_consensus.cc:740] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.726094 28729 ts_tablet_manager.cc:1929] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:22.726260 28729 log.cc:1199] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/026bb078f3904a44820442e69e654f72
I20260320 15:27:22.726754 28729 ts_tablet_manager.cc:1950] T 026bb078f3904a44820442e69e654f72 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:22.726956 28736 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.727470 28734 consensus_queue.cc:260] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.728124 28734 ts_tablet_manager.cc:1434] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:22.731886 28734 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:22.732216 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 45825us
I20260320 15:27:22.733673 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 (DEFAULT_TABLE table=default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.734241 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395977eb0444bcf8563cd8774fab40f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:22.734452 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5438fdc6e6944e79ac8aecd0ef774467. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.734480 28444 raft_consensus.cc:3060] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.736900 28734 tablet_bootstrap.cc:654] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.738040 28444 raft_consensus.cc:2468] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:22.738718 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b395977eb0444bcf8563cd8774fab40f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:22.738957 28587 raft_consensus.cc:3060] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.739502 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.739938 28735 tablet_replica.cc:333] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.740222 28735 raft_consensus.cc:2243] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.740497 28735 raft_consensus.cc:2272] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.741365 28735 ts_tablet_manager.cc:1916] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.741888 28587 raft_consensus.cc:2468] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:22.742501 28239 leader_election.cc:304] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:22.742957 28966 raft_consensus.cc:2804] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:22.743181 28966 raft_consensus.cc:697] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:22.743772 28966 consensus_queue.cc:237] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.744416 28734 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.744607 28734 ts_tablet_manager.cc:1403] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260320 15:27:22.745456 28734 raft_consensus.cc:359] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.745690 28734 raft_consensus.cc:385] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.745797 28734 raft_consensus.cc:740] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.746068 28734 consensus_queue.cc:260] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.747714 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 19c939f8c30943fe8f0619b7dcacc7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.751392 28734 ts_tablet_manager.cc:1434] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260320 15:27:22.752588 28735 ts_tablet_manager.cc:1929] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.752777 28735 log.cc:1199] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:22.753558 28734 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.753999 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 442ca186c9d4449dade39271dec5c608 (DEFAULT_TABLE table=default.Table_eb27eb94efef46238ae0dd4cb3f3bfac [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.754419 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 442ca186c9d4449dade39271dec5c608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.754690 28733 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:22.754951 28733 ts_tablet_manager.cc:1403] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.372s user 0.003s sys 0.010s
I20260320 15:27:22.755509 28734 tablet_bootstrap.cc:654] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.756065 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.756081 28733 raft_consensus.cc:359] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.756309 28733 raft_consensus.cc:385] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.756386 28733 raft_consensus.cc:740] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.756659 28733 consensus_queue.cc:260] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.756914 28730 rpcz_store.cc:276] 0320 15:27:21.664885 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.677905 (+ 13020us) service_pool.cc:225] Handling call
0320 15:27:22.701091 (+1023186us) ts_tablet_manager.cc:1151] Deleting tablet d57566e7907c4783aafce347e46b89e3
0320 15:27:22.701102 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:22.707057 (+ 5955us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1022730}
W20260320 15:27:22.757440 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet d57566e7907c4783aafce347e46b89e3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d57566e7907c4783aafce347e46b89e3 already in progress: creating tablet
I20260320 15:27:22.759578 28730 tablet_replica.cc:333] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:22.759809 28730 raft_consensus.cc:2243] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.761536 28734 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.762945 28734 ts_tablet_manager.cc:1403] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260320 15:27:22.763904 28734 raft_consensus.cc:359] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.764149 28734 raft_consensus.cc:385] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.764252 28734 raft_consensus.cc:740] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.764571 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 9248f0fee2fd446bb8c53891f6e7ba6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.765326 28730 raft_consensus.cc:2272] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.765427 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 026bb078f3904a44820442e69e654f72 with delete_type TABLET_DATA_DELETED (replaced by tablet ca5e0c215f574b8aafec8d5e2da9a399) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.766055 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.766297 28730 ts_tablet_manager.cc:1916] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.766391 28734 consensus_queue.cc:260] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.766773 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:22.766983 28240 leader_election.cc:343] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:22.767213 28240 leader_election.cc:304] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:22.767377 28733 ts_tablet_manager.cc:1434] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260320 15:27:22.768750 28733 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:22.768715 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (DEFAULT_TABLE table=default.Table_a4dec0096d0946b280892bd8fd4ee5a4 [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.769054 28734 ts_tablet_manager.cc:1434] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:22.769519 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6231fe30c5b34d0bb7a233abbffeb1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.775938 28968 raft_consensus.cc:2749] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.776669 28733 tablet_bootstrap.cc:654] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.784315 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.786808 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet f640a6ae307140d2aee3992333e96d84 with delete_type TABLET_DATA_DELETED (replaced by tablet 7feb8477ebf841a19977a434f365f537) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.787473 28734 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.787742 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet f640a6ae307140d2aee3992333e96d84 with delete_type TABLET_DATA_DELETED (replaced by tablet 7feb8477ebf841a19977a434f365f537) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.788287 28736 tablet_bootstrap.cc:654] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.789593 28735 ts_tablet_manager.cc:1950] T 3038fd8c34ee48278404acfec3e0b8d8 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.789803 28734 tablet_bootstrap.cc:654] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.790076 28418 tablet_service.cc:1511] Processing CreateTablet for tablet ca5e0c215f574b8aafec8d5e2da9a399 (DEFAULT_TABLE table=default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.790932 28735 tablet_replica.cc:333] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.790778 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ee4aafbcc4e46bb9dbb0460a8575282 with delete_type TABLET_DATA_DELETED (replaced by tablet 557d8be0e4bf42239b2bb4585e7d0f06) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.791164 28735 raft_consensus.cc:2243] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.791358 28735 raft_consensus.cc:2272] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.792073 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 with delete_type TABLET_DATA_DELETED (replaced by tablet bb7a910bd2904a40ae63608377206c7e) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.792289 28284 tablet_service.cc:1511] Processing CreateTablet for tablet ca5e0c215f574b8aafec8d5e2da9a399 (DEFAULT_TABLE table=default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.792469 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e0c215f574b8aafec8d5e2da9a399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.792742 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca5e0c215f574b8aafec8d5e2da9a399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.794869 28729 tablet_replica.cc:333] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:22.795116 28729 raft_consensus.cc:2243] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.795336 28729 raft_consensus.cc:2272] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.795981 28729 ts_tablet_manager.cc:1916] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.797552 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.796092 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 97811e288242436c8e74fd9f45d8d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.796985 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 2069aa42660947d8a1d37d6d40846991 with delete_type TABLET_DATA_DELETED (replaced by tablet 66a181681cd84b7fb09c3742f4f2222f) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:22.798221 28040 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6231fe30c5b34d0bb7a233abbffeb1e8 consensus state has no leader
I20260320 15:27:22.798610 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.799762 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.800387 28730 ts_tablet_manager.cc:1929] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.800948 28735 ts_tablet_manager.cc:1916] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.802412 28730 log.cc:1199] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/97811e288242436c8e74fd9f45d8d81e
I20260320 15:27:22.802909 28730 ts_tablet_manager.cc:1950] T 97811e288242436c8e74fd9f45d8d81e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:22.809747 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet b2355eadcae6483bae7ae475d74dd131 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:22.809729 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.810158 28049 master_service.cc:954] ReplaceTablet: received request to replace tablet 66a9ddea49e641da881fba339ddef549 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:22.810397 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_38cee1701ad24626b2929b539fdb3c3e" } modify_external_catalogs: true
I20260320 15:27:22.810554 28054 master_service.cc:954] ReplaceTablet: received request to replace tablet 993e2fb1561f4317be85e9b5e5816edc from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:22.810675 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_38cee1701ad24626b2929b539fdb3c3e" } modify_external_catalogs: true
I20260320 15:27:22.810921 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_be682052d66448438f9d7e7fc47a7ced" } modify_external_catalogs: true
I20260320 15:27:22.811152 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_be682052d66448438f9d7e7fc47a7ced" } modify_external_catalogs: true
I20260320 15:27:22.811358 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_99a1f24aff574bff820f96a585a8c417" } modify_external_catalogs: true
I20260320 15:27:22.811573 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_99a1f24aff574bff820f96a585a8c417" } modify_external_catalogs: true
I20260320 15:27:22.814399 28734 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.814615 28734 ts_tablet_manager.cc:1403] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.007s
I20260320 15:27:22.814822 28736 tablet_bootstrap.cc:492] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
W20260320 15:27:22.810485 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7a910bd2904a40ae63608377206c7e already in progress: creating tablet
I20260320 15:27:22.814990 28736 ts_tablet_manager.cc:1403] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.000s
I20260320 15:27:22.815235 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (table default.Table_5abb1b4446804b0eb71ccecaa7dffaf3 [id=41bf3a11683345ff9162410e671ee6ff]) successfully deleted
W20260320 15:27:22.815589 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b395977eb0444bcf8563cd8774fab40f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b395977eb0444bcf8563cd8774fab40f already in progress: creating tablet
W20260320 15:27:22.815838 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b1bc86d3d1ff4b3aa8463eb21f619917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1bc86d3d1ff4b3aa8463eb21f619917 already in progress: creating tablet
I20260320 15:27:22.815871 28734 raft_consensus.cc:359] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.816028 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0ab709738a3f471da773fd9b523f7fbb (table default.Table_1e18f506558543ec8b4ec33dceac311d [id=33d342aac79a4e06ac3660eaff41d38f]) successfully deleted
I20260320 15:27:22.816107 28734 raft_consensus.cc:385] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.816215 28734 raft_consensus.cc:740] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.816543 28734 consensus_queue.cc:260] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.816740 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 3038fd8c34ee48278404acfec3e0b8d8 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
I20260320 15:27:22.819409 28734 ts_tablet_manager.cc:1434] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
W20260320 15:27:22.820314 28383 consensus_peers.cc:597] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:22.820824 28734 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.822913 28734 tablet_bootstrap.cc:654] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.823623 28736 raft_consensus.cc:359] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.823834 28736 raft_consensus.cc:385] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.823953 28736 raft_consensus.cc:740] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.824334 28736 consensus_queue.cc:260] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.818893 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 51ee8b3598384569bbe3d63abb3eaed9 with delete_type TABLET_DATA_DELETED (replaced by tablet bb7a910bd2904a40ae63608377206c7e) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:22.819803 28303 raft_consensus.cc:3060] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:22.825412 28729 ts_tablet_manager.cc:1929] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.825568 28729 log.cc:1199] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0ee4aafbcc4e46bb9dbb0460a8575282
I20260320 15:27:22.826016 28729 ts_tablet_manager.cc:1950] T 0ee4aafbcc4e46bb9dbb0460a8575282 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:22.818768 28587 raft_consensus.cc:1275] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:22.820163 28588 raft_consensus.cc:1275] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:22.816672 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 157) took 1061 ms. Trace:
I20260320 15:27:22.828733 28730 rpcz_store.cc:276] 0320 15:27:21.754981 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.755087 (+ 106us) service_pool.cc:225] Handling call
0320 15:27:22.759472 (+1004385us) ts_tablet_manager.cc:1151] Deleting tablet 97811e288242436c8e74fd9f45d8d81e
0320 15:27:22.759488 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:22.780968 (+ 21480us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:22.800147 (+ 19179us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:22.816592 (+ 16445us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5286,"tablet-delete.queue_time_us":999418}
I20260320 15:27:22.829087 28303 raft_consensus.cc:1275] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:22.829551 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 97811e288242436c8e74fd9f45d8d81e (table default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]) successfully deleted
I20260320 15:27:22.829612 28917 consensus_queue.cc:1048] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:22.830153 28729 tablet_replica.cc:333] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:22.830399 28729 raft_consensus.cc:2243] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.830456 28917 consensus_queue.cc:1048] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:22.830597 28729 raft_consensus.cc:2272] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.831223 28729 ts_tablet_manager.cc:1916] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.831372 28753 consensus_queue.cc:1048] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:22.838883 28736 ts_tablet_manager.cc:1434] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260320 15:27:22.839376 28736 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.841910 28736 tablet_bootstrap.cc:654] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.845666 28735 ts_tablet_manager.cc:1929] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.845875 28735 log.cc:1199] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/19c939f8c30943fe8f0619b7dcacc7e0
I20260320 15:27:22.846298 28735 ts_tablet_manager.cc:1950] T 19c939f8c30943fe8f0619b7dcacc7e0 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.849351 28734 tablet_bootstrap.cc:492] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.851717 28734 ts_tablet_manager.cc:1403] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.006s
I20260320 15:27:22.853735 28734 raft_consensus.cc:359] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.853988 28734 raft_consensus.cc:385] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.854079 28734 raft_consensus.cc:740] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.856743 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 159) took 1071 ms. Trace:
I20260320 15:27:22.856900 28730 rpcz_store.cc:276] 0320 15:27:21.772299 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.772501 (+ 202us) service_pool.cc:225] Handling call
0320 15:27:22.829084 (+1056583us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:22.829097 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:22.843836 (+ 14739us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1056304}
I20260320 15:27:22.857882 28730 tablet_replica.cc:333] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:22.858121 28730 raft_consensus.cc:2243] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.858352 28730 raft_consensus.cc:2272] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.858609 28753 raft_consensus.cc:493] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.858810 28753 raft_consensus.cc:515] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.859262 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 66a9ddea49e641da881fba339ddef549
I20260320 15:27:22.859930 28049 catalog_manager.cc:6741] ReplaceTablet: tablet 66a9ddea49e641da881fba339ddef549 deleted and replaced by tablet 4becc525b59d421caf79d8cd42052118
I20260320 15:27:22.860255 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993c9b6ae14ae98c2239abd65333ea" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:22.861052 28383 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.861672 28753 leader_election.cc:290] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:22.861753 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 5bc3fb47136d4b88a2898af399d74a8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bc3fb47136d4b88a2898af399d74a8a already in progress: creating tablet
I20260320 15:27:22.862413 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:22.863435 28382 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:22.863619 28382 leader_election.cc:304] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:22.864081 28730 ts_tablet_manager.cc:1916] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.864179 28753 raft_consensus.cc:2749] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.866658 28733 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:22.866989 28733 ts_tablet_manager.cc:1403] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.098s user 0.010s sys 0.005s
I20260320 15:27:22.869185 28733 raft_consensus.cc:359] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.870474 28735 tablet_replica.cc:333] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:22.870792 28735 raft_consensus.cc:2243] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.871047 28735 raft_consensus.cc:2272] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.872354 28735 ts_tablet_manager.cc:1916] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.874738 28045 catalog_manager.cc:5671] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:22.875110 28729 ts_tablet_manager.cc:1929] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.875284 28729 log.cc:1199] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:22.875272 28045 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:22.875734 28729 ts_tablet_manager.cc:1950] T 51ee8b3598384569bbe3d63abb3eaed9 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:22.875921 28733 raft_consensus.cc:385] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.876145 28733 raft_consensus.cc:740] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.876533 28736 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:22.876724 28736 ts_tablet_manager.cc:1403] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.000s
I20260320 15:27:22.876952 28733 consensus_queue.cc:260] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.877748 28736 raft_consensus.cc:359] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.878005 28736 raft_consensus.cc:385] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.878072 28736 raft_consensus.cc:740] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.878342 28736 consensus_queue.cc:260] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.879298 28733 ts_tablet_manager.cc:1434] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260320 15:27:22.880977 28736 ts_tablet_manager.cc:1434] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:22.881376 28736 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:22.881872 28859 raft_consensus.cc:493] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.882028 28859 raft_consensus.cc:515] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.882717 28859 leader_election.cc:290] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.883167 28733 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:22.884784 28239 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:22.876483 28734 consensus_queue.cc:260] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.882982 28920 raft_consensus.cc:493] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:22.888571 28920 raft_consensus.cc:515] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.889415 28920 leader_election.cc:290] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:22.883875 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51ee8b3598384569bbe3d63abb3eaed9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:22.890997 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a1e0957439458083e3a86ac2f01b1e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:22.891662 28240 leader_election.cc:343] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260320 15:27:22.891866 28240 leader_election.cc:304] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:22.891894 28383 leader_election.cc:343] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.892179 28859 raft_consensus.cc:2749] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.893231 28979 mvcc.cc:204] Tried to move back new op lower bound from 7266387733782474752 to 7266387732948066304. Current Snapshot: MvccSnapshot[applied={T|T < 7266387733782474752}]
I20260320 15:27:22.893960 28734 ts_tablet_manager.cc:1434] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20260320 15:27:22.896023 28736 tablet_bootstrap.cc:654] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.896369 28047 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]). Expected version 1 got 0
I20260320 15:27:22.899863 28734 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.903594 28047 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]). Expected version 1 got 0
I20260320 15:27:22.904098 28047 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:22.904807 28734 tablet_bootstrap.cc:654] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.906122 28729 tablet_replica.cc:333] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:22.906438 28729 raft_consensus.cc:2243] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:22.887715 28733 tablet_bootstrap.cc:654] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.907819 28729 raft_consensus.cc:2272] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:22.890461 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a1e0957439458083e3a86ac2f01b1e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:22.909477 28734 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.909658 28734 ts_tablet_manager.cc:1403] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.002s
I20260320 15:27:22.910759 28734 raft_consensus.cc:359] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.911000 28734 raft_consensus.cc:385] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.911118 28734 raft_consensus.cc:740] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.911176 28045 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:22.911394 28734 consensus_queue.cc:260] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.913151 28734 ts_tablet_manager.cc:1434] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:22.914172 28734 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.916672 28734 tablet_bootstrap.cc:654] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.922353 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:22.922923 28054 catalog_manager.cc:6741] ReplaceTablet: tablet 993e2fb1561f4317be85e9b5e5816edc deleted and replaced by tablet 583ed053b2a742c7b029f348f4495101
I20260320 15:27:22.923491 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 757b5afdea674217909610e3719a8804
I20260320 15:27:22.930512 28729 ts_tablet_manager.cc:1916] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:22.931427 28734 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.931663 28734 ts_tablet_manager.cc:1403] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.003s
I20260320 15:27:22.932636 28734 raft_consensus.cc:359] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.932873 28734 raft_consensus.cc:385] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.932966 28734 raft_consensus.cc:740] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.933295 28734 consensus_queue.cc:260] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.936265 28734 ts_tablet_manager.cc:1434] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:22.936678 28734 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.936838 28991 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260320 15:27:22.938318 28997 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.62:43951 every 8 connection(s)
I20260320 15:27:22.941084 28734 tablet_bootstrap.cc:654] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.942261 28992 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260320 15:27:22.943689 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 3038fd8c34ee48278404acfec3e0b8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.944629 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:22.945564 28744 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.62:43951
W20260320 15:27:22.946511 28382 leader_election.cc:343] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:22.946689 28382 leader_election.cc:304] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:22.947584 28920 raft_consensus.cc:2749] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:22.948253 28563 tablet_service.cc:1511] Processing CreateTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.949156 28556 tablet_service.cc:1511] Processing CreateTablet for tablet f3e4a749f0744e68b502af97edeeb584 (DEFAULT_TABLE table=default.Table_c19d1d44b172435dbb61e970e1b3ced2 [id=c825e1c06ebc423c98aadd08b90bfdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:22.949292 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.949548 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:22.949868 28300 raft_consensus.cc:1275] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:22.950054 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bfb87d5d5741adb0ad2f2a88f4867f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.950347 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e4a749f0744e68b502af97edeeb584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:22.950812 28586 raft_consensus.cc:1275] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:22.952069 28920 consensus_queue.cc:1048] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:22.960565 28753 consensus_queue.cc:1048] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:22.966476 28734 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:22.966856 28734 ts_tablet_manager.cc:1403] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.001s
I20260320 15:27:22.968148 28734 raft_consensus.cc:359] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.968493 28734 raft_consensus.cc:385] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:22.968672 28734 raft_consensus.cc:740] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:22.969048 28734 consensus_queue.cc:260] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:22.969938 28734 ts_tablet_manager.cc:1434] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:22.970393 28734 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:22.972327 29001 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:22.972651 28734 tablet_bootstrap.cc:654] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:22.974373 28992 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:22.976570 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]) successfully deleted
W20260320 15:27:22.977048 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7394e941c99e4358ad7a33db84b15348 already in progress: creating tablet
W20260320 15:27:22.977469 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adb65832eefe4bd4b08d95aa83c7bd7b already in progress: creating tablet
W20260320 15:27:22.977820 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f640a6ae307140d2aee3992333e96d84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f640a6ae307140d2aee3992333e96d84 already in progress: creating tablet
I20260320 15:27:22.978063 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0ee4aafbcc4e46bb9dbb0460a8575282 (table default.Table_5abb1b4446804b0eb71ccecaa7dffaf3 [id=41bf3a11683345ff9162410e671ee6ff]) successfully deleted
I20260320 15:27:22.979601 28744 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-0/data/info.pb
I20260320 15:27:22.983918 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 28744
I20260320 15:27:22.985226 28735 ts_tablet_manager.cc:1929] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.985765 28735 log.cc:1199] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9248f0fee2fd446bb8c53891f6e7ba6e
I20260320 15:27:22.986274 28735 ts_tablet_manager.cc:1950] T 9248f0fee2fd446bb8c53891f6e7ba6e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:22.979418 28730 ts_tablet_manager.cc:1929] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:22.993124 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 51ee8b3598384569bbe3d63abb3eaed9 (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
I20260320 15:27:22.997603 28047 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
I20260320 15:27:22.998301 28047 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]). Expected version 1 got 0
I20260320 15:27:23.000701 28736 tablet_bootstrap.cc:492] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.000797 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.000891 28736 ts_tablet_manager.cc:1403] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.120s user 0.009s sys 0.007s
I20260320 15:27:23.001811 28736 raft_consensus.cc:359] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.002048 28736 raft_consensus.cc:385] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.002147 28736 raft_consensus.cc:740] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.003362 28736 consensus_queue.cc:260] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.006194 28736 ts_tablet_manager.cc:1434] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:23.006676 28736 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.008867 28736 tablet_bootstrap.cc:654] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.013870 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b6201b74fdb0428e98f2eebb0b24d01a
I20260320 15:27:23.016521 28917 raft_consensus.cc:493] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.016848 28917 raft_consensus.cc:515] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.018859 28917 leader_election.cc:290] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.019369 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14be8e6ed89b413a84f9517ca5ff02dc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.019562 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:23.020267 28383 leader_election.cc:343] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.021597 28045 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:23.023876 28735 tablet_replica.cc:333] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.025192 28735 raft_consensus.cc:2243] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.025607 28735 raft_consensus.cc:2272] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.030129 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:23.031111 28048 catalog_manager.cc:6741] ReplaceTablet: tablet b2355eadcae6483bae7ae475d74dd131 deleted and replaced by tablet 9176e734212849aab8d99473de02cc79
I20260320 15:27:23.032588 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.034138 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:23.026415 28735 ts_tablet_manager.cc:1916] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.031896 28350 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:23.035943 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.036134 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.038511 28730 log.cc:1199] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2069aa42660947d8a1d37d6d40846991
I20260320 15:27:23.032796 28736 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.041558 28736 ts_tablet_manager.cc:1403] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.005s
W20260320 15:27:23.042503 28382 leader_election.cc:343] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:23.042672 28382 leader_election.cc:304] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.043431 28917 raft_consensus.cc:2749] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.045447 28736 raft_consensus.cc:359] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.045710 28736 raft_consensus.cc:385] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.045609 28556 tablet_service.cc:1511] Processing CreateTablet for tablet b4cad900411e4c9686701aa99206dd23 (DEFAULT_TABLE table=default.Table_b91605000af04532ba5d3fc895d8dc1a [id=7b608418906d4f4596846ba336c3c4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.045784 28736 raft_consensus.cc:740] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.047729 28730 ts_tablet_manager.cc:1950] T 2069aa42660947d8a1d37d6d40846991 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:23.048537 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.049337 28736 consensus_queue.cc:260] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.049912 28047 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343] alter complete (version 5)
I20260320 15:27:23.050107 28736 ts_tablet_manager.cc:1434] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:23.050437 28736 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.052424 28736 tablet_bootstrap.cc:654] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.052826 28734 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:23.053022 28734 ts_tablet_manager.cc:1403] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.083s user 0.002s sys 0.006s
I20260320 15:27:23.053930 28734 raft_consensus.cc:359] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.054726 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.056768 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.057588 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.057898 28493 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:23.058115 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.059628 28047 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_248df89af9b1443e8ef145d0cf33ae62 [id=399646419f3b4fd28458144f43bd4553] alter complete (version 2)
I20260320 15:27:23.063186 28052 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:23.066479 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.067960 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.070384 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.054158 28734 raft_consensus.cc:385] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.071436 28734 raft_consensus.cc:740] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.073738 28274 tablet_service.cc:1511] Processing CreateTablet for tablet f3e4a749f0744e68b502af97edeeb584 (DEFAULT_TABLE table=default.Table_c19d1d44b172435dbb61e970e1b3ced2 [id=c825e1c06ebc423c98aadd08b90bfdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.074294 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e4a749f0744e68b502af97edeeb584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.074201 28734 consensus_queue.cc:260] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.075090 28736 tablet_bootstrap.cc:492] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.075273 28736 ts_tablet_manager.cc:1403] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.009s
I20260320 15:27:23.076236 28736 raft_consensus.cc:359] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.076460 28736 raft_consensus.cc:385] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.076556 28736 raft_consensus.cc:740] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.076805 28736 consensus_queue.cc:260] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.077299 28734 ts_tablet_manager.cc:1434] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260320 15:27:23.077687 28734 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:23.078300 28736 ts_tablet_manager.cc:1434] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:23.078649 28736 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.080737 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed29146bff74e86a366d3c6329176f4 (DEFAULT_TABLE table=default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.081418 28634 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:23.083101 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.083601 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4cad900411e4c9686701aa99206dd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.084085 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 160) took 1311 ms. Trace:
I20260320 15:27:23.086354 28493 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:23.087433 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed29146bff74e86a366d3c6329176f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.088966 28549 tablet_service.cc:1511] Processing CreateTablet for tablet ce8595157679444b959b569dbb7d8df0 (DEFAULT_TABLE table=default.Table_d3fccc4dd32348d1addde0067aeae899 [id=75466621ee1941378a891a480a3df191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.091050 28418 tablet_service.cc:1511] Processing CreateTablet for tablet f3e4a749f0744e68b502af97edeeb584 (DEFAULT_TABLE table=default.Table_c19d1d44b172435dbb61e970e1b3ced2 [id=c825e1c06ebc423c98aadd08b90bfdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.095907 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e4a749f0744e68b502af97edeeb584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.095924 28274 tablet_service.cc:1511] Processing CreateTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.097127 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_adb981152b5c4e7793de15d294bb7f34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.098011 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bfb87d5d5741adb0ad2f2a88f4867f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.100412 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adb981152b5c4e7793de15d294bb7f34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.104200 29005 sys_catalog.cc:263] Verifying existing consensus state
I20260320 15:27:23.104867 28735 ts_tablet_manager.cc:1929] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.105059 28735 log.cc:1199] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/026bb078f3904a44820442e69e654f72
I20260320 15:27:23.105511 28735 ts_tablet_manager.cc:1950] T 026bb078f3904a44820442e69e654f72 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:23.106729 28735 tablet_replica.cc:333] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.106952 28735 raft_consensus.cc:2243] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.107182 28735 raft_consensus.cc:2272] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.108469 28736 tablet_bootstrap.cc:654] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.108414 28735 ts_tablet_manager.cc:1916] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.114811 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.092968 28917 raft_consensus.cc:493] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.121996 28735 ts_tablet_manager.cc:1929] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.122246 28735 log.cc:1199] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f640a6ae307140d2aee3992333e96d84
I20260320 15:27:23.122068 28917 raft_consensus.cc:515] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.122728 28735 ts_tablet_manager.cc:1950] T f640a6ae307140d2aee3992333e96d84 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:23.123842 28917 leader_election.cc:290] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.093726 28425 tablet_service.cc:1511] Processing CreateTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.124761 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97811e288242436c8e74fd9f45d8d81e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.124964 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4bfb87d5d5741adb0ad2f2a88f4867f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.125490 28383 leader_election.cc:343] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.127130 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_af7bda6fe8e14815bfd5bf848ac65d4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.127439 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5d5d276ee59e4bc68cecddd3cff48146" } modify_external_catalogs: true
I20260320 15:27:23.127712 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5d5d276ee59e4bc68cecddd3cff48146" } modify_external_catalogs: true
W20260320 15:27:23.128036 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.130779 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c77304a1097d4fb987cc1aa19d0fb284"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.131415 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c77304a1097d4fb987cc1aa19d0fb284 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.133299 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_98404b0cbf764bed9e21a83afa8a7dcd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.133718 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_051864d5d9984140b3ce79ec28717420"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.133935 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98404b0cbf764bed9e21a83afa8a7dcd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:23.134238 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_051864d5d9984140b3ce79ec28717420 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.137940 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 906f207dee674a4288c8c60b5d3ff9f8
I20260320 15:27:23.139130 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_613a6835f9f44c3392f000337fb9dfdd" } new_table_name: "default.Table_30015eb07aba409bb03a37140eea20d6" modify_external_catalogs: true
I20260320 15:27:23.140844 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3979e77f6aba4d9f80ba813c92287b84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.141813 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3979e77f6aba4d9f80ba813c92287b84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.142135 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_8a5cb4ae3fa742009b86bfba15cd4b41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.143026 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a5cb4ae3fa742009b86bfba15cd4b41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.143141 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 19c939f8c30943fe8f0619b7dcacc7e0 (table default.Table_adca08e7f2314aec8ad43dc91db184de [id=43686d94fa59452b8aa697f3d7b63cf7]) successfully deleted
I20260320 15:27:23.143404 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9248f0fee2fd446bb8c53891f6e7ba6e (table default.Table_55de4adf4e2242bf959513e74510cb9d [id=948843ec6d3a419789304be665f78107]) successfully deleted
I20260320 15:27:23.143803 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]) successfully deleted
W20260320 15:27:23.144094 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7394e941c99e4358ad7a33db84b15348 already in progress: creating tablet
I20260320 15:27:23.146371 29005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Bootstrap starting.
I20260320 15:27:23.146317 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c0173c5015ce4d06aa17f7cbb3ecd511" } new_table_name: "default.Table_6892575e01594ece9a265373580a05bc" modify_external_catalogs: true
I20260320 15:27:23.146737 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet f640a6ae307140d2aee3992333e96d84 with delete_type TABLET_DATA_DELETED (replaced by tablet 7feb8477ebf841a19977a434f365f537) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.147992 28270 tablet_service.cc:1511] Processing CreateTablet for tablet b4cad900411e4c9686701aa99206dd23 (DEFAULT_TABLE table=default.Table_b91605000af04532ba5d3fc895d8dc1a [id=7b608418906d4f4596846ba336c3c4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.148237 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.148324 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.148567 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.148767 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.149302 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.149981 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.150906 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.151440 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.151902 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.152010 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4b73c356945a441bb0fc92e4ac7e3f6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.152671 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b73c356945a441bb0fc92e4ac7e3f6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.153291 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed29146bff74e86a366d3c6329176f4 (DEFAULT_TABLE table=default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.153739 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed29146bff74e86a366d3c6329176f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.154174 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed29146bff74e86a366d3c6329176f4 (DEFAULT_TABLE table=default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.143055 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_83f833bca071404790ce7888026891b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.156589 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83f833bca071404790ce7888026891b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.157958 28425 tablet_service.cc:1511] Processing CreateTablet for tablet b4cad900411e4c9686701aa99206dd23 (DEFAULT_TABLE table=default.Table_b91605000af04532ba5d3fc895d8dc1a [id=7b608418906d4f4596846ba336c3c4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.158435 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4cad900411e4c9686701aa99206dd23. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.161713 28382 leader_election.cc:343] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 97811e288242436c8e74fd9f45d8d81e
I20260320 15:27:23.161872 28382 leader_election.cc:304] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.162560 28917 raft_consensus.cc:2749] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.162935 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f325d0702e60408fb5a7d3a21290c416" } modify_external_catalogs: true
I20260320 15:27:23.163182 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f325d0702e60408fb5a7d3a21290c416" } modify_external_catalogs: true
I20260320 15:27:23.163369 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e4cb7c7492b24f5caedd6760d05e6190" } new_table_name: "default.Table_17c9ae5b739344f4ab272260023b6a74" modify_external_catalogs: true
I20260320 15:27:23.164639 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b993c9b6ae14ae98c2239abd65333ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.165140 28272 tablet_service.cc:1511] Processing CreateTablet for tablet ce8595157679444b959b569dbb7d8df0 (DEFAULT_TABLE table=default.Table_d3fccc4dd32348d1addde0067aeae899 [id=75466621ee1941378a891a480a3df191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.165310 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.165766 28284 tablet_service.cc:1511] Processing CreateTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (DEFAULT_TABLE table=default.Table_613a6835f9f44c3392f000337fb9dfdd [id=83739ddc977f4089b85ad8e6ea64df8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.165944 28414 tablet_service.cc:1511] Processing CreateTablet for tablet ce8595157679444b959b569dbb7d8df0 (DEFAULT_TABLE table=default.Table_d3fccc4dd32348d1addde0067aeae899 [id=75466621ee1941378a891a480a3df191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.166245 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.166435 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8595157679444b959b569dbb7d8df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.169032 28427 tablet_service.cc:1511] Processing CreateTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (DEFAULT_TABLE table=default.Table_613a6835f9f44c3392f000337fb9dfdd [id=83739ddc977f4089b85ad8e6ea64df8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.169517 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecaa08ba34514c5e99a9e2d4ba9268a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.171329 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.172067 28936 raft_consensus.cc:493] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.172304 28936 raft_consensus.cc:515] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.174341 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26818e620ff649d18b671e4f4c7c7e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.175122 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.175737 28553 tablet_service.cc:1511] Processing CreateTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (DEFAULT_TABLE table=default.Table_613a6835f9f44c3392f000337fb9dfdd [id=83739ddc977f4089b85ad8e6ea64df8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.175882 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.178191 28277 tablet_service.cc:1511] Processing CreateTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 (DEFAULT_TABLE table=default.Table_e4cb7c7492b24f5caedd6760d05e6190 [id=30b8ab71f82a42ebbb2976696e1ecb4e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:23.180289 28524 leader_election.cc:343] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.181891 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.095438 28729 ts_tablet_manager.cc:1929] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.182205 28729 log.cc:1199] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:23.182157 28417 tablet_service.cc:1511] Processing CreateTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 (DEFAULT_TABLE table=default.Table_e4cb7c7492b24f5caedd6760d05e6190 [id=30b8ab71f82a42ebbb2976696e1ecb4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.092942 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.101922 28920 raft_consensus.cc:493] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.182487 28920 raft_consensus.cc:515] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.182691 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7bd0616f5604f8fbb8edd8e9c230f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.182862 28444 raft_consensus.cc:2468] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:23.183329 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.183403 28920 leader_election.cc:290] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.121605 28734 tablet_bootstrap.cc:654] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.186514 28729 ts_tablet_manager.cc:1950] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:23.188090 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.188683 28735 tablet_replica.cc:333] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.189000 28735 raft_consensus.cc:2243] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.189204 28735 raft_consensus.cc:2272] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.190361 28735 ts_tablet_manager.cc:1916] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.190707 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8595157679444b959b569dbb7d8df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.190754 28524 leader_election.cc:304] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:23.191144 28936 leader_election.cc:290] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:23.191345 28936 raft_consensus.cc:2804] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.191458 28936 raft_consensus.cc:493] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.191573 28936 raft_consensus.cc:3060] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.189044 28383 leader_election.cc:343] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.194566 28936 raft_consensus.cc:515] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.195221 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.195742 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26818e620ff649d18b671e4f4c7c7e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.198741 28730 rpcz_store.cc:276] 0320 15:27:21.772438 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.772823 (+ 385us) service_pool.cc:225] Handling call
0320 15:27:22.857374 (+1084551us) ts_tablet_manager.cc:1151] Deleting tablet 2069aa42660947d8a1d37d6d40846991
0320 15:27:22.857774 (+ 400us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:22.868842 (+ 11068us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:22.954391 (+ 85549us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.084034 (+129643us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1084363}
I20260320 15:27:23.199522 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.199927 28585 raft_consensus.cc:2393] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:23.201867 28736 tablet_bootstrap.cc:492] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.202067 28736 ts_tablet_manager.cc:1403] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.123s user 0.003s sys 0.014s
I20260320 15:27:23.202337 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecaa08ba34514c5e99a9e2d4ba9268a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.202997 28736 raft_consensus.cc:359] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.203269 28736 raft_consensus.cc:385] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.203361 28736 raft_consensus.cc:740] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.203864 28561 tablet_service.cc:1511] Processing CreateTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 (DEFAULT_TABLE table=default.Table_e4cb7c7492b24f5caedd6760d05e6190 [id=30b8ab71f82a42ebbb2976696e1ecb4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.204557 28735 ts_tablet_manager.cc:1929] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.204700 28735 log.cc:1199] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/51ee8b3598384569bbe3d63abb3eaed9
I20260320 15:27:23.205238 28735 ts_tablet_manager.cc:1950] T 51ee8b3598384569bbe3d63abb3eaed9 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:23.205636 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 158) took 1429 ms. Trace:
I20260320 15:27:23.206353 28552 tablet_service.cc:1511] Processing CreateTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 (DEFAULT_TABLE table=default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.207052 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 2069aa42660947d8a1d37d6d40846991 (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:23.211658 28382 leader_election.cc:304] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.215225 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ed5104d8fb4f435e87290e89a34f96e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.216188 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed5104d8fb4f435e87290e89a34f96e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.219964 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a719aa4da12a4d159826d615a6a2302a" } new_table_name: "default.Table_874d6a667bff4c3d815fcff90cae13f5" modify_external_catalogs: true
I20260320 15:27:23.222592 28736 consensus_queue.cc:260] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.223336 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.224383 28736 ts_tablet_manager.cc:1434] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260320 15:27:23.224617 28735 tablet_replica.cc:333] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.224879 28735 raft_consensus.cc:2243] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:23.225255 28735 pending_rounds.cc:70] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:23.225322 28735 pending_rounds.cc:77] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387733484830720 op_type: NO_OP noop_request { }
I20260320 15:27:23.225466 28735 raft_consensus.cc:2889] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:23.225616 28735 raft_consensus.cc:2272] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.226083 28736 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.227597 28753 raft_consensus.cc:493] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.228207 28736 tablet_bootstrap.cc:654] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.231045 28281 tablet_service.cc:1511] Processing CreateTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 (DEFAULT_TABLE table=default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.231660 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea95b7a1b30e42cebcbfcd551c58f679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.234330 28936 leader_election.cc:290] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:23.238118 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7bd0616f5604f8fbb8edd8e9c230f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.241183 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (DEFAULT_TABLE table=default.Table_4fa9df7189e84297bb2b7b04ae79052f [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.241793 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.244108 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (DEFAULT_TABLE table=default.Table_4fa9df7189e84297bb2b7b04ae79052f [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.244884 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 7feb8477ebf841a19977a434f365f537 (DEFAULT_TABLE table=default.Table_d27df02667144e339d2b08cacb4b5e36 [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.245404 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7feb8477ebf841a19977a434f365f537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.246206 28729 tablet_replica.cc:333] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:23.247758 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet f640a6ae307140d2aee3992333e96d84 (table default.Table_d27df02667144e339d2b08cacb4b5e36 [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
W20260320 15:27:23.248121 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adb65832eefe4bd4b08d95aa83c7bd7b already in progress: creating tablet
I20260320 15:27:23.248113 28753 raft_consensus.cc:515] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.248306 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 51ee8b3598384569bbe3d63abb3eaed9 (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
W20260320 15:27:23.248487 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 66a9ddea49e641da881fba339ddef549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a9ddea49e641da881fba339ddef549 already in progress: creating tablet
W20260320 15:27:23.248643 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 993e2fb1561f4317be85e9b5e5816edc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 993e2fb1561f4317be85e9b5e5816edc already in progress: creating tablet
W20260320 15:27:23.248803 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 757b5afdea674217909610e3719a8804 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 757b5afdea674217909610e3719a8804 already in progress: creating tablet
I20260320 15:27:23.248852 28753 leader_election.cc:290] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.249161 28920 raft_consensus.cc:3060] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.249330 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (table default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]) successfully deleted
I20260320 15:27:23.249578 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.249811 28303 raft_consensus.cc:2468] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:23.250711 28383 leader_election.cc:304] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:23.253368 28917 raft_consensus.cc:493] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.255602 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.255867 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (DEFAULT_TABLE table=default.Table_4fa9df7189e84297bb2b7b04ae79052f [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.256384 28730 rpcz_store.cc:276] 0320 15:27:21.772158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.773039 (+ 881us) service_pool.cc:225] Handling call
0320 15:27:23.199122 (+1426083us) ts_tablet_manager.cc:1151] Deleting tablet 557d8be0e4bf42239b2bb4585e7d0f06
0320 15:27:23.199135 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.201105 (+ 1970us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1425929}
W20260320 15:27:23.257524 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 557d8be0e4bf42239b2bb4585e7d0f06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 557d8be0e4bf42239b2bb4585e7d0f06 already in progress: creating tablet
I20260320 15:27:23.258545 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 163) took 1437 ms. Trace:
I20260320 15:27:23.258749 28730 rpcz_store.cc:276] 0320 15:27:21.821153 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.821211 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:23.258270 (+1437059us) ts_tablet_manager.cc:1151] Deleting tablet 9aa2d48a54dd403c82d7f3a64db5aeab
0320 15:27:23.258282 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.258476 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1435266}
I20260320 15:27:23.259056 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.259590 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 165) took 1432 ms. Trace:
I20260320 15:27:23.259968 28730 rpcz_store.cc:276] 0320 15:27:21.826772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.826835 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:23.259316 (+1432481us) ts_tablet_manager.cc:1151] Deleting tablet da957aed322f46acaf5876c567c2ff94
0320 15:27:23.259347 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.259521 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1429803}
I20260320 15:27:23.260968 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 166) took 1433 ms. Trace:
I20260320 15:27:23.261186 28730 rpcz_store.cc:276] 0320 15:27:21.826915 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.826959 (+ 44us) service_pool.cc:225] Handling call
0320 15:27:23.260553 (+1433594us) ts_tablet_manager.cc:1151] Deleting tablet 6d1db9963c4645ae9aab72b4b22d6938
0320 15:27:23.260566 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.260753 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1430871}
I20260320 15:27:23.262300 28736 tablet_bootstrap.cc:492] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.262581 28736 ts_tablet_manager.cc:1403] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.000s
W20260320 15:27:23.262570 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9aa2d48a54dd403c82d7f3a64db5aeab already in progress: creating tablet
W20260320 15:27:23.263051 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet da957aed322f46acaf5876c567c2ff94 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da957aed322f46acaf5876c567c2ff94 already in progress: creating tablet
I20260320 15:27:23.263216 28859 raft_consensus.cc:493] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.263382 28859 raft_consensus.cc:515] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:23.263520 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 6d1db9963c4645ae9aab72b4b22d6938 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d1db9963c4645ae9aab72b4b22d6938 already in progress: creating tablet
I20260320 15:27:23.264184 28859 leader_election.cc:290] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.265256 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.265508 28585 raft_consensus.cc:2468] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:23.267122 28917 raft_consensus.cc:515] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.267599 28239 leader_election.cc:304] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:23.267764 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.268018 28859 raft_consensus.cc:2804] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.268143 28859 raft_consensus.cc:493] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.268237 28859 raft_consensus.cc:3060] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.269604 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 7feb8477ebf841a19977a434f365f537 (DEFAULT_TABLE table=default.Table_d27df02667144e339d2b08cacb4b5e36 [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.270170 28920 raft_consensus.cc:2749] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.271602 28859 raft_consensus.cc:515] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.272961 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:23.273614 28240 leader_election.cc:343] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.273602 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 164) took 1446 ms. Trace:
I20260320 15:27:23.273880 28753 raft_consensus.cc:2804] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.273819 28730 rpcz_store.cc:276] 0320 15:27:21.826656 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.827240 (+ 584us) service_pool.cc:225] Handling call
0320 15:27:23.262016 (+1434776us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:23.262027 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.273483 (+ 11456us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1432194}
I20260320 15:27:23.274688 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 173) took 1378 ms. Trace:
I20260320 15:27:23.274904 28730 rpcz_store.cc:276] 0320 15:27:21.895779 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:21.895881 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:23.274368 (+1378487us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:23.274409 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.274619 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1339129}
I20260320 15:27:23.275841 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 185) took 1091 ms. Trace:
I20260320 15:27:23.276154 28730 rpcz_store.cc:276] 0320 15:27:22.183939 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.191614 (+ 7675us) service_pool.cc:225] Handling call
0320 15:27:23.275582 (+1083968us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:23.275592 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.275774 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1070210}
I20260320 15:27:23.277225 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 186) took 1079 ms. Trace:
I20260320 15:27:23.277595 28730 rpcz_store.cc:276] 0320 15:27:22.197867 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.212224 (+ 14357us) service_pool.cc:225] Handling call
0320 15:27:23.276861 (+1064637us) ts_tablet_manager.cc:1151] Deleting tablet da860b779a174adbb4fd6f7f7520348c
0320 15:27:23.276911 (+ 50us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.277140 (+ 229us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1064196}
I20260320 15:27:23.278844 28753 raft_consensus.cc:493] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.283895 28859 leader_election.cc:290] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:23.284440 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c323b7bcb671464ea6ff584a633f10c0 already in progress: creating tablet
I20260320 15:27:23.284492 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:23.284721 28585 raft_consensus.cc:3060] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.284878 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b82c47ae3786401d931bb0b486ca3570 already in progress: creating tablet
W20260320 15:27:23.285105 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b03e7555b9a646b599acb0c7dff90c9a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b03e7555b9a646b599acb0c7dff90c9a already in progress: creating tablet
W20260320 15:27:23.285466 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet da860b779a174adbb4fd6f7f7520348c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da860b779a174adbb4fd6f7f7520348c already in progress: creating tablet
I20260320 15:27:23.285940 28736 raft_consensus.cc:359] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.286156 28736 raft_consensus.cc:385] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.286260 28736 raft_consensus.cc:740] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.287590 28736 consensus_queue.cc:260] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.288296 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.288205 28427 tablet_service.cc:1511] Processing CreateTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 (DEFAULT_TABLE table=default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.288439 28585 raft_consensus.cc:2468] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:23.288746 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "825f7e3fdc004fe1a3bcca6deb20cccd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.289112 28239 leader_election.cc:304] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:23.289381 28240 leader_election.cc:343] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.289489 28917 leader_election.cc:290] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:23.289819 28383 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:23.290067 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7feb8477ebf841a19977a434f365f537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.291078 28859 raft_consensus.cc:2804] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.291226 28859 raft_consensus.cc:697] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:23.291582 28859 consensus_queue.cc:237] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.294317 28920 raft_consensus.cc:493] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.294497 28920 raft_consensus.cc:515] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.295228 28920 leader_election.cc:290] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.296433 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e8f37a490a4fdfbc2a3dc56f6da297" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.297439 28753 raft_consensus.cc:3060] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.298750 28383 leader_election.cc:343] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.299058 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea95b7a1b30e42cebcbfcd551c58f679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.300091 28303 raft_consensus.cc:3060] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.300335 28736 ts_tablet_manager.cc:1434] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.037s user 0.002s sys 0.000s
I20260320 15:27:23.300765 28736 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.303839 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4cad900411e4c9686701aa99206dd23. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.294974 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 111242us
I20260320 15:27:23.305568 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.298274 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea95b7a1b30e42cebcbfcd551c58f679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.307960 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed29146bff74e86a366d3c6329176f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.308027 28736 tablet_bootstrap.cc:654] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.310844 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:23.311887 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce8595157679444b959b569dbb7d8df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.315973 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecaa08ba34514c5e99a9e2d4ba9268a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.319485 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26818e620ff649d18b671e4f4c7c7e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.321342 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 7feb8477ebf841a19977a434f365f537 (DEFAULT_TABLE table=default.Table_d27df02667144e339d2b08cacb4b5e36 [id=4d6287f82e004f15b09b7afa88fb47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.322568 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.325124 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7bd0616f5604f8fbb8edd8e9c230f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.324903 28936 raft_consensus.cc:493] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.326118 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.329001 28735 ts_tablet_manager.cc:1916] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.334767 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e8f37a490a4fdfbc2a3dc56f6da297" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.335556 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7feb8477ebf841a19977a434f365f537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.337553 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 189) took 1060 ms. Trace:
I20260320 15:27:23.337704 28730 rpcz_store.cc:276] 0320 15:27:22.276546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.276779 (+ 233us) service_pool.cc:225] Handling call
0320 15:27:23.330040 (+1053261us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:23.330055 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.337423 (+ 7368us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":988393}
W20260320 15:27:23.338261 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7a910bd2904a40ae63608377206c7e already in progress: creating tablet
I20260320 15:27:23.352533 28729 raft_consensus.cc:2243] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.353453 28936 raft_consensus.cc:515] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:23.355278 28382 leader_election.cc:343] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:23.355818 28382 leader_election.cc:304] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.356118 28917 raft_consensus.cc:2749] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:23.356421 28382 leader_election.cc:343] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.356586 28382 leader_election.cc:304] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.357302 28729 raft_consensus.cc:2272] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.357661 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b6165eab6b0442fa9d94ce577c1b43c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:23.359925 28920 raft_consensus.cc:670] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260320 15:27:23.360818 29001 tablet.cc:1723] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:23.360819 28729 ts_tablet_manager.cc:1916] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.360931 28920 raft_consensus.cc:2749] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.373885 28444 raft_consensus.cc:2468] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:23.377025 28753 raft_consensus.cc:515] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.379561 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:23.379844 28300 raft_consensus.cc:3060] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.380695 28734 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:23.380959 28734 ts_tablet_manager.cc:1403] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.303s user 0.016s sys 0.016s
I20260320 15:27:23.381347 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.382083 28734 raft_consensus.cc:359] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.383901 28734 raft_consensus.cc:385] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.384341 28734 raft_consensus.cc:740] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.384456 28585 raft_consensus.cc:2468] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:23.384990 28734 consensus_queue.cc:260] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.387935 28300 raft_consensus.cc:2468] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:23.392949 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ecaa08ba34514c5e99a9e2d4ba9268a9 consensus state has no leader
I20260320 15:27:23.393633 28383 leader_election.cc:304] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:23.394024 28920 raft_consensus.cc:2804] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.394152 28920 raft_consensus.cc:697] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:23.394460 28920 consensus_queue.cc:237] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:23.394881 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ca38bc990634e21a2c38dd840488f8c consensus state has no leader
I20260320 15:27:23.403601 28753 leader_election.cc:290] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.413300 29005 log.cc:826] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:23.415252 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.426149 28859 raft_consensus.cc:493] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.426785 28859 raft_consensus.cc:515] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.434693 28936 leader_election.cc:290] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:23.395314 28524 leader_election.cc:343] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.440352 28047 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:23.441681 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3a07fada18374bf297e97be27789813f" } modify_external_catalogs: true
I20260320 15:27:23.441937 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3a07fada18374bf297e97be27789813f" } modify_external_catalogs: true
I20260320 15:27:23.447265 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 1268b25a1c99404aa77e5b1bcc415f48 with delete_type TABLET_DATA_DELETED (replaced by tablet 89425da569794a8fa972817023bf853d) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.447224 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7c20a59bd0eb41739dfa0770f3cff465" } modify_external_catalogs: true
I20260320 15:27:23.449251 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 191) took 1044 ms. Trace:
I20260320 15:27:23.449635 28730 rpcz_store.cc:276] 0320 15:27:22.344554 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.345020 (+ 466us) service_pool.cc:225] Handling call
0320 15:27:23.347520 (+1002500us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:23.347558 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.388729 (+ 41171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":995563}
W20260320 15:27:23.450421 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b1bc86d3d1ff4b3aa8463eb21f619917 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1bc86d3d1ff4b3aa8463eb21f619917 already in progress: creating tablet
I20260320 15:27:23.450959 28730 tablet_replica.cc:333] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:23.451216 28730 raft_consensus.cc:2243] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.451612 28730 raft_consensus.cc:2272] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.451249 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cfc52407bf394111b223a209b9adbdec" } modify_external_catalogs: true
I20260320 15:27:23.451784 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_cfc52407bf394111b223a209b9adbdec" } modify_external_catalogs: true
W20260320 15:27:23.449098 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 124432us
I20260320 15:27:23.452982 28524 leader_election.cc:304] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:23.454630 28992 tablet.cc:1723] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:23.455824 28936 raft_consensus.cc:2804] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.455811 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.456015 28936 raft_consensus.cc:697] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:23.456487 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.456570 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1f7a90e189124cc49bcbd37f8df5a793" } new_table_name: "default.Table_e25b7a014de24f8c8f22a3584fd0d367" modify_external_catalogs: true
I20260320 15:27:23.456595 28936 consensus_queue.cc:237] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.455427 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.458608 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7c20a59bd0eb41739dfa0770f3cff465" } modify_external_catalogs: true
I20260320 15:27:23.458941 28300 raft_consensus.cc:2393] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:23.460843 28524 leader_election.cc:304] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:23.463927 28966 raft_consensus.cc:493] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.464141 28966 raft_consensus.cc:515] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.464983 28966 leader_election.cc:290] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.465418 28729 ts_tablet_manager.cc:1929] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.465677 28729 log.cc:1199] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2069aa42660947d8a1d37d6d40846991
I20260320 15:27:23.466086 28729 ts_tablet_manager.cc:1950] T 2069aa42660947d8a1d37d6d40846991 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:23.466590 28239 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:23.467372 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 2069aa42660947d8a1d37d6d40846991 (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:23.469136 28729 tablet_replica.cc:333] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:23.470734 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993c9b6ae14ae98c2239abd65333ea" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.470995 28444 raft_consensus.cc:2468] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:23.471684 28240 leader_election.cc:304] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.473182 28966 raft_consensus.cc:2804] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.473524 28966 raft_consensus.cc:493] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.473716 28966 raft_consensus.cc:3060] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.478547 28733 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:23.478755 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.479847 28733 ts_tablet_manager.cc:1403] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.597s user 0.031s sys 0.015s
I20260320 15:27:23.480224 29018 raft_consensus.cc:3060] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.483160 28859 leader_election.cc:290] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.484030 29009 mvcc.cc:204] Tried to move back new op lower bound from 7266387734309367808 to 7266387732381917184. Current Snapshot: MvccSnapshot[applied={T|T < 7266387734309367808}]
W20260320 15:27:23.484319 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 127892us
I20260320 15:27:23.484586 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.484910 28585 raft_consensus.cc:2468] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:23.486881 28239 leader_election.cc:304] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:23.486048 28733 raft_consensus.cc:359] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.487275 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1268b25a1c99404aa77e5b1bcc415f48" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:23.487764 28733 raft_consensus.cc:385] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.488194 28733 raft_consensus.cc:740] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.489024 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.489395 28444 raft_consensus.cc:2468] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:23.493036 28966 raft_consensus.cc:515] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.495199 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993c9b6ae14ae98c2239abd65333ea" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:23.495558 28444 raft_consensus.cc:3060] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.496184 28239 leader_election.cc:343] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:23.496394 29026 raft_consensus.cc:493] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.496562 29026 raft_consensus.cc:515] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.497678 29026 leader_election.cc:290] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.498620 28734 ts_tablet_manager.cc:1434] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.117s user 0.001s sys 0.001s
I20260320 15:27:23.498688 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.500334 28444 raft_consensus.cc:2468] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:23.501094 28240 leader_election.cc:304] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.502277 29035 raft_consensus.cc:2804] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.502498 29035 raft_consensus.cc:493] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.502657 29035 raft_consensus.cc:3060] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.502682 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7bd0616f5604f8fbb8edd8e9c230f39 consensus state has no leader
I20260320 15:27:23.507184 28859 raft_consensus.cc:2804] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.509858 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ce76eeddb583441bb18fa5fb8793ffb0
I20260320 15:27:23.512894 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.513069 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.514559 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c77304a1097d4fb987cc1aa19d0fb284" } modify_external_catalogs: true
I20260320 15:27:23.514824 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c77304a1097d4fb987cc1aa19d0fb284" } modify_external_catalogs: true
I20260320 15:27:23.515564 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d1d5feef4fbc48b08f2deb15f2f277f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.516105 28588 raft_consensus.cc:2393] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
W20260320 15:27:23.516386 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.518357 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e9195525674d41a89bd4e262baa86a81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.519040 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9195525674d41a89bd4e262baa86a81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.521387 28730 ts_tablet_manager.cc:1916] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.521809 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:23.524000 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a072859f39c4c28ae65b0e9afa79d66 consensus state has no leader
I20260320 15:27:23.524580 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_26547ac0e535447fbdbc2aa66519b8fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.525301 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26547ac0e535447fbdbc2aa66519b8fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.531409 28382 leader_election.cc:304] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.509004 28966 leader_election.cc:290] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.513990 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet ce76eeddb583441bb18fa5fb8793ffb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.516959 29035 raft_consensus.cc:515] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.534132 29035 leader_election.cc:290] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.493036 28733 consensus_queue.cc:260] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.540087 28859 raft_consensus.cc:697] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:23.540560 28859 consensus_queue.cc:237] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.534140 28729 raft_consensus.cc:2243] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.537035 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:23.544713 28444 raft_consensus.cc:3060] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.550459 28444 raft_consensus.cc:2468] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:23.543467 28734 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:23.553757 28734 tablet_bootstrap.cc:654] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:23.555994 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 95154us
I20260320 15:27:23.559034 29018 raft_consensus.cc:2749] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.551573 28240 leader_election.cc:304] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:23.559864 29036 raft_consensus.cc:2804] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.560281 29036 raft_consensus.cc:697] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:23.560758 29036 consensus_queue.cc:237] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.565140 29035 raft_consensus.cc:493] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.565392 29035 raft_consensus.cc:515] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.566533 29035 leader_election.cc:290] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.555804 28736 tablet_bootstrap.cc:492] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.569424 28736 ts_tablet_manager.cc:1403] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.269s user 0.060s sys 0.125s
W20260320 15:27:23.570924 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 931f6f88a1d94a40b69f07216b8313c8 consensus state has no leader
I20260320 15:27:23.571559 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_750f7d0833f145d391cdda2b262f13ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.572440 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_750f7d0833f145d391cdda2b262f13ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.573463 28736 raft_consensus.cc:359] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.573729 28736 raft_consensus.cc:385] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.573813 28736 raft_consensus.cc:740] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.574781 28736 consensus_queue.cc:260] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.578224 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_9ee2dbc7d4f343f2b58a9fee8543896a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.579645 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:23.580680 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce76eeddb583441bb18fa5fb8793ffb0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:23.582264 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.582975 28588 raft_consensus.cc:3060] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:23.583384 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ee2dbc7d4f343f2b58a9fee8543896a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.585630 28736 ts_tablet_manager.cc:1434] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260320 15:27:23.586143 28736 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.586668 29036 raft_consensus.cc:493] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.586854 29036 raft_consensus.cc:515] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:23.586938 29020 log.cc:927] Time spent T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260320 15:27:23.588591 29036 leader_election.cc:290] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:23.589186 28240 leader_election.cc:343] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 2069aa42660947d8a1d37d6d40846991
I20260320 15:27:23.589504 28736 tablet_bootstrap.cc:654] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.607352 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c57985f062354b93bd560e9d7012fe5e" } new_table_name: "default.Table_a15c29714da04984898578d391977a08" modify_external_catalogs: true
I20260320 15:27:23.608995 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 (DEFAULT_TABLE table=default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.609125 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9c0b3e51cc4042fab00ee1662295787e" } new_table_name: "default.Table_6462b376d8f34deba33a10a982b4fed4" modify_external_catalogs: true
I20260320 15:27:23.609064 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f4751ac3d9fd417ba487d086d653b85d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.609715 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a4e61bab44d768d9adfed00c945e9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.609903 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4751ac3d9fd417ba487d086d653b85d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.614189 28048 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]). Expected version 1 got 0
I20260320 15:27:23.614580 28048 catalog_manager.cc:5671] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:23.616323 29033 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:23.622720 28053 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e] alter complete (version 1)
I20260320 15:27:23.555325 29026 raft_consensus.cc:3060] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.627281 28585 raft_consensus.cc:2468] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:23.627882 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.629994 29026 raft_consensus.cc:2749] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.557101 28729 raft_consensus.cc:2272] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.582430 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545c8fd2d4c845adba6dbf42dfddba3e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:23.582993 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.588104 28917 raft_consensus.cc:493] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.630661 28917 raft_consensus.cc:515] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.631067 28413 tablet_service.cc:1511] Processing CreateTablet for tablet ba4eb20eb0f1403282030727757031f7 (DEFAULT_TABLE table=default.Table_98404b0cbf764bed9e21a83afa8a7dcd [id=4654af060c0c4bc19f8d93dba5db8dd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.631520 28917 leader_election.cc:290] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.631606 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4eb20eb0f1403282030727757031f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.601675 29025 raft_consensus.cc:493] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.631768 29025 raft_consensus.cc:515] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.632423 29025 leader_election.cc:290] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.633199 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 583ed053b2a742c7b029f348f4495101 (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.633699 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583ed053b2a742c7b029f348f4495101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.634689 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 4becc525b59d421caf79d8cd42052118 (DEFAULT_TABLE table=default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.635216 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 9176e734212849aab8d99473de02cc79 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.635401 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4becc525b59d421caf79d8cd42052118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.635690 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9176e734212849aab8d99473de02cc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.635952 28272 tablet_service.cc:1511] Processing CreateTablet for tablet ba4eb20eb0f1403282030727757031f7 (DEFAULT_TABLE table=default.Table_98404b0cbf764bed9e21a83afa8a7dcd [id=4654af060c0c4bc19f8d93dba5db8dd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.636464 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 583ed053b2a742c7b029f348f4495101 (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.636974 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 (DEFAULT_TABLE table=default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.637380 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a0fadc7dac4e21ad034e4cd2d70826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.640435 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2548d7b8d31049519a85490772621352" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:23.608398 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 (DEFAULT_TABLE table=default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.641733 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a4e61bab44d768d9adfed00c945e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.642664 28729 ts_tablet_manager.cc:1916] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:23.642902 28240 leader_election.cc:343] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.644045 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 (DEFAULT_TABLE table=default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.644676 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 4becc525b59d421caf79d8cd42052118 (DEFAULT_TABLE table=default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.645088 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 9176e734212849aab8d99473de02cc79 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.647992 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a341aa1a6fbe4dedb77cedb3339ea59d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:23.647945 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.648733 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:23.648835 28383 leader_election.cc:343] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: f640a6ae307140d2aee3992333e96d84
W20260320 15:27:23.649120 28383 leader_election.cc:343] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:23.651340 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a341aa1a6fbe4dedb77cedb3339ea59d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.654249 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4eb20eb0f1403282030727757031f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.654768 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.655797 29041 tablet.cc:1723] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:23.621999 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545c8fd2d4c845adba6dbf42dfddba3e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.663059 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583ed053b2a742c7b029f348f4495101. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.663925 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 (DEFAULT_TABLE table=default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.668601 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a0fadc7dac4e21ad034e4cd2d70826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.672004 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4becc525b59d421caf79d8cd42052118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.672626 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9176e734212849aab8d99473de02cc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.691582 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a4e61bab44d768d9adfed00c945e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.692225 29005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Bootstrap replayed 1/1 log segments. Stats: ops{read=29 overwritten=0 applied=26 ignored=0} inserts{seen=23 ignored=0} mutations{seen=17 ignored=0} orphaned_commits=0. Pending: 3 replicates
I20260320 15:27:23.692190 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.693040 29005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Bootstrap complete.
I20260320 15:27:23.697979 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 583ed053b2a742c7b029f348f4495101 (DEFAULT_TABLE table=default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.700505 28550 tablet_service.cc:1511] Processing CreateTablet for tablet ba4eb20eb0f1403282030727757031f7 (DEFAULT_TABLE table=default.Table_98404b0cbf764bed9e21a83afa8a7dcd [id=4654af060c0c4bc19f8d93dba5db8dd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.702253 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 (DEFAULT_TABLE table=default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.702832 28549 tablet_service.cc:1511] Processing CreateTablet for tablet 4becc525b59d421caf79d8cd42052118 (DEFAULT_TABLE table=default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.706995 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 583ed053b2a742c7b029f348f4495101. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:23.712383 28239 leader_election.cc:343] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 2069aa42660947d8a1d37d6d40846991
I20260320 15:27:23.712924 28239 leader_election.cc:304] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:23.713851 28859 raft_consensus.cc:2749] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:23.713984 28239 leader_election.cc:343] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.714154 28239 leader_election.cc:304] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:23.715770 28859 raft_consensus.cc:2749] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.715095 29005 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 FOLLOWER]: Replica starting. Triggering 3 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:23.723054 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4becc525b59d421caf79d8cd42052118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.724807 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43a0fadc7dac4e21ad034e4cd2d70826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.728421 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba4eb20eb0f1403282030727757031f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.737007 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 9176e734212849aab8d99473de02cc79 (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:23.737229 28735 ts_tablet_manager.cc:1929] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:23.737618 28735 log.cc:1199] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:23.737648 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9176e734212849aab8d99473de02cc79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:23.738435 28735 ts_tablet_manager.cc:1950] T b395977eb0444bcf8563cd8774fab40f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:23.743476 28734 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:23.743748 28734 ts_tablet_manager.cc:1403] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.245s user 0.005s sys 0.001s
I20260320 15:27:23.744659 28734 raft_consensus.cc:359] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.744867 28734 raft_consensus.cc:385] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.744933 28734 raft_consensus.cc:740] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.745309 28734 consensus_queue.cc:260] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.746380 28734 ts_tablet_manager.cc:1434] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:23.746939 28734 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:23.760448 28729 ts_tablet_manager.cc:1929] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:23.764607 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5a20d7f22cfa403184e73b24d565b60a consensus state has no leader
I20260320 15:27:23.765409 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:23.766150 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 66a181681cd84b7fb09c3742f4f2222f from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:23.768033 28733 ts_tablet_manager.cc:1434] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.286s user 0.017s sys 0.017s
I20260320 15:27:23.769665 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2548d7b8d31049519a85490772621352" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.769757 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 0e9a4e61bab44d768d9adfed00c945e9
W20260320 15:27:23.770269 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45a0e8e2ce7a454ebcab4e59a4de68dd consensus state has no leader
I20260320 15:27:23.772119 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b395977eb0444bcf8563cd8774fab40f (table default.Table_bce978bcf8414d928d5f77eaf4e90cbe [id=95ea602214894df19afd570e9cf0258d]) successfully deleted
I20260320 15:27:23.765643 28735 tablet_replica.cc:333] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.776491 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.776808 28859 consensus_queue.cc:1048] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:23.791893 28584 raft_consensus.cc:2468] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:23.780045 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.784838 28350 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:23.793677 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.797674 29036 consensus_queue.cc:1048] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:23.799005 28735 raft_consensus.cc:2243] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.801330 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.805956 28735 raft_consensus.cc:2272] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.807410 28735 ts_tablet_manager.cc:1916] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.808377 29045 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260320 15:27:23.810207 28733 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:23.774603 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.779655 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.812000 28382 leader_election.cc:304] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:23.813537 29025 raft_consensus.cc:2804] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:23.813716 29025 raft_consensus.cc:493] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:23.813813 29025 raft_consensus.cc:3060] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.814527 28729 log.cc:1199] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/97811e288242436c8e74fd9f45d8d81e
W20260320 15:27:23.816962 28039 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_a15c29714da04984898578d391977a08 [id=a010951cfbe240fea932452776c43ad6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5a20d7f22cfa403184e73b24d565b60a consensus state has no leader
I20260320 15:27:23.801107 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f640a6ae307140d2aee3992333e96d84" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:23.817886 29025 raft_consensus.cc:515] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.818238 28729 ts_tablet_manager.cc:1950] T 97811e288242436c8e74fd9f45d8d81e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:23.819257 29025 leader_election.cc:290] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:23.819504 29005 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.820293 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2548d7b8d31049519a85490772621352" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:23.820792 29005 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 3.29, Last appended by leader: 29, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
W20260320 15:27:23.821033 28383 leader_election.cc:343] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.820806 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 202) took 1024 ms. Trace:
I20260320 15:27:23.824313 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 583ed053b2a742c7b029f348f4495101
I20260320 15:27:23.823570 28729 rpcz_store.cc:276] 0320 15:27:22.795927 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.795991 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:23.469020 (+673029us) ts_tablet_manager.cc:1151] Deleting tablet 97811e288242436c8e74fd9f45d8d81e
0320 15:27:23.469036 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.646236 (+177200us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.760245 (+114009us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.820601 (+ 60356us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":65855,"tablet-delete.queue_time_us":670991}
I20260320 15:27:23.826511 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.827646 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.827965 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 97811e288242436c8e74fd9f45d8d81e (table default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]) successfully deleted
I20260320 15:27:23.827869 28588 raft_consensus.cc:2468] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
W20260320 15:27:23.834020 28382 leader_election.cc:343] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:23.834127 28729 tablet_replica.cc:333] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:23.835098 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:23.834391 28382 leader_election.cc:304] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:23.837231 28729 raft_consensus.cc:2243] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.840754 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.842310 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 14be8e6ed89b413a84f9517ca5ff02dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.843825 28050 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_ac4367958ed24d109d328c0c2d1098aa [id=634e24fdc81640fbb393a4b19bac7343]). Expected version 5 got 0
I20260320 15:27:23.844116 28050 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_248df89af9b1443e8ef145d0cf33ae62 [id=399646419f3b4fd28458144f43bd4553]). Expected version 2 got 0
I20260320 15:27:23.844492 29025 raft_consensus.cc:2749] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:23.845078 28729 raft_consensus.cc:2272] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.846705 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2548d7b8d31049519a85490772621352" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:23.848119 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4b73c356945a441bb0fc92e4ac7e3f6d" } new_table_name: "default.Table_864c3add59ac47ec8bbbcb4f4796beee" modify_external_catalogs: true
I20260320 15:27:23.848749 28729 ts_tablet_manager.cc:1916] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.850699 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f9e940b8c37f46a6a671a47344d1c74d" } modify_external_catalogs: true
I20260320 15:27:23.850916 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f9e940b8c37f46a6a671a47344d1c74d" } modify_external_catalogs: true
I20260320 15:27:23.852202 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.852854 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.853263 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.855096 28588 raft_consensus.cc:3060] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:23.856112 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_591ab13c4a4a4aa9ba82b371492cab9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.856771 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_591ab13c4a4a4aa9ba82b371492cab9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.859652 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4068b4f067d74effb3bc0dc5bb1faddc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.860266 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4068b4f067d74effb3bc0dc5bb1faddc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.860417 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:23.863675 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4165a4481d3840ceb77849e30d57927d" } new_table_name: "default.Table_ff78e3b872da42d8a80c4592a3f702b2" modify_external_catalogs: true
I20260320 15:27:23.865079 29005 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: configured and running, proceeding with master startup.
I20260320 15:27:23.868013 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_50e316065f1e43459edfe03e6c2468b7" } new_table_name: "default.Table_4389a8ff013e4930b2ebd4703a35f0da" modify_external_catalogs: true
I20260320 15:27:23.876788 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_6cadeb80b9df43149080a924bb38ae8e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.877771 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cadeb80b9df43149080a924bb38ae8e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:23.880084 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 2069aa42660947d8a1d37d6d40846991 with delete_type TABLET_DATA_DELETED (replaced by tablet 66a181681cd84b7fb09c3742f4f2222f) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:23.882097 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_051864d5d9984140b3ce79ec28717420" } modify_external_catalogs: true
I20260320 15:27:23.882356 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_051864d5d9984140b3ce79ec28717420" } modify_external_catalogs: true
I20260320 15:27:23.887576 28730 ts_tablet_manager.cc:1929] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.887854 28730 log.cc:1199] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:23.894017 28730 ts_tablet_manager.cc:1950] T b395977eb0444bcf8563cd8774fab40f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:23.894336 28735 ts_tablet_manager.cc:1929] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.894527 28735 log.cc:1199] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/545c8fd2d4c845adba6dbf42dfddba3e
I20260320 15:27:23.894805 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 190) took 1550 ms. Trace:
I20260320 15:27:23.894970 28735 ts_tablet_manager.cc:1950] T 545c8fd2d4c845adba6dbf42dfddba3e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:23.895058 28730 rpcz_store.cc:276] 0320 15:27:22.344260 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.345454 (+ 1194us) service_pool.cc:225] Handling call
0320 15:27:23.450858 (+1105404us) ts_tablet_manager.cc:1151] Deleting tablet b395977eb0444bcf8563cd8774fab40f
0320 15:27:23.450872 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.827561 (+376689us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.880941 (+ 53380us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.894710 (+ 13769us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1069374}
I20260320 15:27:23.899750 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b395977eb0444bcf8563cd8774fab40f (table default.Table_bce978bcf8414d928d5f77eaf4e90cbe [id=95ea602214894df19afd570e9cf0258d]) successfully deleted
I20260320 15:27:23.901124 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.906133 28588 raft_consensus.cc:2468] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:23.907028 28382 leader_election.cc:304] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:23.907625 29025 raft_consensus.cc:2804] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:23.907819 29025 raft_consensus.cc:697] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:23.908140 29025 consensus_queue.cc:237] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.914850 28733 tablet_bootstrap.cc:654] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.916815 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 545c8fd2d4c845adba6dbf42dfddba3e (table default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]) successfully deleted
I20260320 15:27:23.923691 28730 tablet_replica.cc:333] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:23.924537 28730 raft_consensus.cc:2243] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:23.924995 28730 raft_consensus.cc:2272] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:23.925987 28730 ts_tablet_manager.cc:1916] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:23.926146 28734 tablet_bootstrap.cc:654] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.927487 28736 tablet_bootstrap.cc:492] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:23.927732 28736 ts_tablet_manager.cc:1403] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.342s user 0.049s sys 0.080s
I20260320 15:27:23.928925 28736 raft_consensus.cc:359] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.929141 28736 raft_consensus.cc:385] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.929279 28736 raft_consensus.cc:740] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.929836 28736 consensus_queue.cc:260] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.930682 28736 ts_tablet_manager.cc:1434] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:23.931066 28736 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:23.933568 28736 tablet_bootstrap.cc:654] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:23.934893 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.949661 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.949915 28927 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 3 FOLLOWER]: Advancing to term 4
I20260320 15:27:23.953320 28735 tablet_replica.cc:333] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:23.953596 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:23.953759 28735 raft_consensus.cc:2243] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:23.954077 28735 pending_rounds.cc:70] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:23.954182 28735 pending_rounds.cc:77] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387736832032768 op_type: NO_OP noop_request { }
I20260320 15:27:23.954327 28735 raft_consensus.cc:2889] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:23.954615 28735 raft_consensus.cc:2272] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:23.955473 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6201b74fdb0428e98f2eebb0b24d01a already in progress: creating tablet
W20260320 15:27:23.956209 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b2355eadcae6483bae7ae475d74dd131 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2355eadcae6483bae7ae475d74dd131 already in progress: creating tablet
W20260320 15:27:23.956699 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c752fb99ef494bda9d144d0e180e3100 already in progress: creating tablet
I20260320 15:27:23.957302 28734 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:23.957585 28734 ts_tablet_manager.cc:1403] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.211s user 0.005s sys 0.001s
I20260320 15:27:23.958066 28729 ts_tablet_manager.cc:1929] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.958309 28729 log.cc:1199] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:23.958871 28729 ts_tablet_manager.cc:1950] T 9b993c9b6ae14ae98c2239abd65333ea P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:23.959048 28734 raft_consensus.cc:359] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.964901 28733 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:23.965227 28733 ts_tablet_manager.cc:1403] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.195s user 0.003s sys 0.007s
I20260320 15:27:23.966867 28733 raft_consensus.cc:359] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.968333 28733 raft_consensus.cc:385] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.969094 28733 raft_consensus.cc:740] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.969633 28733 consensus_queue.cc:260] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.972904 28733 ts_tablet_manager.cc:1434] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:23.974300 28733 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:23.974270 28734 raft_consensus.cc:385] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:23.974452 28734 raft_consensus.cc:740] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:23.976640 28730 ts_tablet_manager.cc:1929] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:23.977283 28730 log.cc:1199] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/026bb078f3904a44820442e69e654f72
I20260320 15:27:23.974812 28734 consensus_queue.cc:260] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.990471 28734 ts_tablet_manager.cc:1434] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
I20260320 15:27:23.990589 29035 raft_consensus.cc:493] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:23.990823 29035 raft_consensus.cc:515] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:23.990953 28054 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:23.991146 28734 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:23.992239 29035 leader_election.cc:290] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:23.994103 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ba738a76a424c18b36985a1977bcee4 consensus state has no leader
I20260320 15:27:23.995180 28734 tablet_bootstrap.cc:654] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:23.996312 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c6f8c20b5904eafb9a62d14bb155c82 consensus state has no leader
I20260320 15:27:23.996802 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:23.997722 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.000399 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.001288 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_94aca51eb5d64663a167b3806116218e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.002022 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94aca51eb5d64663a167b3806116218e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:24.003594 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef09e4afdd1f4fce9caec4dcafa2c50c has no consensus state
I20260320 15:27:24.004081 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.004773 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.009716 28735 ts_tablet_manager.cc:1916] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:24.011840 28239 leader_election.cc:343] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:23.959395 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 205) took 1161 ms. Trace:
I20260320 15:27:24.013712 28729 rpcz_store.cc:276] 0320 15:27:22.798327 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.798498 (+ 171us) service_pool.cc:225] Handling call
0320 15:27:23.832898 (+1034400us) ts_tablet_manager.cc:1151] Deleting tablet 9b993c9b6ae14ae98c2239abd65333ea
0320 15:27:23.833572 (+ 674us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.890073 (+ 56501us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.957898 (+ 67825us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.959327 (+ 1429us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1032351}
I20260320 15:27:24.016048 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet 7f0242a7968541f480367b45cda5d9cc from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.018167 28729 tablet_replica.cc:333] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.018577 28729 raft_consensus.cc:2243] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:24.019047 28729 pending_rounds.cc:70] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:24.019219 28729 pending_rounds.cc:77] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387736147001344 op_type: NO_OP noop_request { }
I20260320 15:27:24.019481 28729 raft_consensus.cc:2889] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:24.020215 28729 raft_consensus.cc:2272] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.020932 28736 tablet_bootstrap.cc:492] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.021153 28736 ts_tablet_manager.cc:1403] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.090s user 0.015s sys 0.020s
I20260320 15:27:24.022089 28736 raft_consensus.cc:359] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.022390 28736 raft_consensus.cc:385] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.022481 28736 raft_consensus.cc:740] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.022856 28736 consensus_queue.cc:260] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.023013 28054 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:24.023263 28729 ts_tablet_manager.cc:1916] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.024073 28048 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]). Expected version 1 got 0
I20260320 15:27:24.024796 28736 ts_tablet_manager.cc:1434] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260320 15:27:24.026098 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 2165371f4b5448c7a2625926082ee4c6 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.030066 28734 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.030453 28734 ts_tablet_manager.cc:1403] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.007s
I20260320 15:27:24.031885 28736 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.033344 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14be8e6ed89b413a84f9517ca5ff02dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:24.033383 28734 raft_consensus.cc:359] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.035732 28444 raft_consensus.cc:2468] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:24.035908 28734 raft_consensus.cc:385] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.036196 28734 raft_consensus.cc:740] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.036842 28734 consensus_queue.cc:260] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.037590 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.037603 28240 leader_election.cc:304] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:24.037967 29035 raft_consensus.cc:2804] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.038086 29035 raft_consensus.cc:493] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.038185 29035 raft_consensus.cc:3060] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.037665 28734 ts_tablet_manager.cc:1434] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:24.038630 28734 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:24.041333 29035 raft_consensus.cc:515] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.042333 29035 leader_election.cc:290] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:24.044368 28239 leader_election.cc:343] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:24.044919 28734 tablet_bootstrap.cc:654] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.045277 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.046183 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14be8e6ed89b413a84f9517ca5ff02dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:24.046583 28444 raft_consensus.cc:3060] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.051041 28444 raft_consensus.cc:2468] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:24.052088 28729 ts_tablet_manager.cc:1929] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:24.052312 28729 log.cc:1199] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1268b25a1c99404aa77e5b1bcc415f48
I20260320 15:27:24.052484 28240 leader_election.cc:304] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:24.052875 28729 ts_tablet_manager.cc:1950] T 1268b25a1c99404aa77e5b1bcc415f48 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.053052 29035 raft_consensus.cc:2804] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:24.053335 29035 raft_consensus.cc:697] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:24.053678 29035 consensus_queue.cc:237] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.053961 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 206) took 1254 ms. Trace:
I20260320 15:27:24.058965 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.059792 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 97811e288242436c8e74fd9f45d8d81e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.059914 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.067282 28734 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.069414 28736 tablet_bootstrap.cc:654] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.069737 28734 ts_tablet_manager.cc:1403] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.002s
I20260320 15:27:24.070748 28734 raft_consensus.cc:359] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.070972 28734 raft_consensus.cc:385] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.071055 28734 raft_consensus.cc:740] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.071362 28734 consensus_queue.cc:260] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.072053 28734 ts_tablet_manager.cc:1434] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:24.072726 28734 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:24.073216 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet b82c47ae3786401d931bb0b486ca3570 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.074084 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a30a799b46f848b98910771717144df3" } modify_external_catalogs: true
I20260320 15:27:24.074350 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a30a799b46f848b98910771717144df3" } modify_external_catalogs: true
I20260320 15:27:24.074755 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.075277 28734 tablet_bootstrap.cc:654] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.054103 28729 rpcz_store.cc:276] 0320 15:27:22.799637 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.799686 (+ 49us) service_pool.cc:225] Handling call
0320 15:27:24.018029 (+1218343us) ts_tablet_manager.cc:1151] Deleting tablet 1268b25a1c99404aa77e5b1bcc415f48
0320 15:27:24.018042 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.032114 (+ 14072us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.051823 (+ 19709us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.053892 (+ 2069us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1218052}
I20260320 15:27:24.076017 28729 tablet_replica.cc:333] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.076233 28729 raft_consensus.cc:2243] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.076414 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8a5cb4ae3fa742009b86bfba15cd4b41" } new_table_name: "default.Table_dd29804c0ff542cf8f2d36b963880b98" modify_external_catalogs: true
I20260320 15:27:24.076467 28729 raft_consensus.cc:2272] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.077613 28729 ts_tablet_manager.cc:1916] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:24.087400 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 20a7164db1e0447fb424d8e46b747f52 has no consensus state
I20260320 15:27:24.094079 28729 ts_tablet_manager.cc:1929] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.095145 28729 log.cc:1199] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/3038fd8c34ee48278404acfec3e0b8d8
I20260320 15:27:24.096100 28729 ts_tablet_manager.cc:1950] T 3038fd8c34ee48278404acfec3e0b8d8 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.096315 28927 pending_rounds.cc:85] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Aborting all ops after (but not including) 26
I20260320 15:27:24.096741 28927 pending_rounds.cc:107] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Aborting uncommitted WRITE_OP operation due to leader change: 3.27
I20260320 15:27:24.097571 28927 pending_rounds.cc:107] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Aborting uncommitted WRITE_OP operation due to leader change: 3.28
I20260320 15:27:24.098137 28927 pending_rounds.cc:107] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Aborting uncommitted WRITE_OP operation due to leader change: 3.29
I20260320 15:27:24.098304 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 209) took 1154 ms. Trace:
I20260320 15:27:24.098513 28729 rpcz_store.cc:276] 0320 15:27:22.943487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.943573 (+ 86us) service_pool.cc:225] Handling call
0320 15:27:24.075921 (+1132348us) ts_tablet_manager.cc:1151] Deleting tablet 3038fd8c34ee48278404acfec3e0b8d8
0320 15:27:24.075933 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.090190 (+ 14257us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.093909 (+ 3719us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.098237 (+ 4328us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6400,"tablet-delete.queue_time_us":1131989}
I20260320 15:27:24.099064 28729 tablet_replica.cc:333] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.101511 28735 ts_tablet_manager.cc:1929] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:24.101552 28729 raft_consensus.cc:2243] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.101817 28735 log.cc:1199] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9b993c9b6ae14ae98c2239abd65333ea
I20260320 15:27:24.101828 28729 raft_consensus.cc:2272] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.102771 28730 ts_tablet_manager.cc:1950] T 026bb078f3904a44820442e69e654f72 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:24.008020 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.103344 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 199) took 1624 ms. Trace:
I20260320 15:27:24.103602 28730 rpcz_store.cc:276] 0320 15:27:22.479360 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.479440 (+ 80us) service_pool.cc:225] Handling call
0320 15:27:23.921775 (+1442335us) ts_tablet_manager.cc:1151] Deleting tablet 026bb078f3904a44820442e69e654f72
0320 15:27:23.923581 (+ 1806us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:23.930703 (+ 7122us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:23.963173 (+ 32470us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.103287 (+140114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1442001}
I20260320 15:27:24.104368 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 026bb078f3904a44820442e69e654f72 (table default.Table_04b04acd60a042188e87b5d447398dce [id=f9156a7fb60142b886e96708d989e7e2]) successfully deleted
I20260320 15:27:24.105168 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 200) took 1573 ms. Trace:
I20260320 15:27:24.105594 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9b993c9b6ae14ae98c2239abd65333ea (table default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]) successfully deleted
I20260320 15:27:24.105945 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1268b25a1c99404aa77e5b1bcc415f48 (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:24.106266 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 3038fd8c34ee48278404acfec3e0b8d8 (table default.Table_066f88ea62914d6f93053e540c6eae61 [id=4494ebcc34f1468c9ce7381b5e5ab7d6]) successfully deleted
I20260320 15:27:24.108031 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_750f7d0833f145d391cdda2b262f13ed" } new_table_name: "default.Table_9234513d2cd54f55bf709a1c6fb0de1c" modify_external_catalogs: true
I20260320 15:27:24.112418 28729 ts_tablet_manager.cc:1916] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.115231 28730 rpcz_store.cc:276] 0320 15:27:22.531611 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.531876 (+ 265us) service_pool.cc:225] Handling call
0320 15:27:24.104945 (+1573069us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:24.104959 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.105117 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1564265}
I20260320 15:27:24.116134 28735 ts_tablet_manager.cc:1950] T 9b993c9b6ae14ae98c2239abd65333ea P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.118079 28736 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.118295 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_adb981152b5c4e7793de15d294bb7f34" } modify_external_catalogs: true
I20260320 15:27:24.118507 28736 ts_tablet_manager.cc:1403] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.087s user 0.004s sys 0.001s
I20260320 15:27:24.118607 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_adb981152b5c4e7793de15d294bb7f34" } modify_external_catalogs: true
I20260320 15:27:24.120229 28736 raft_consensus.cc:359] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.120579 28736 raft_consensus.cc:385] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:24.120579 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7394e941c99e4358ad7a33db84b15348 already in progress: creating tablet
I20260320 15:27:24.120733 28736 raft_consensus.cc:740] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.120994 28736 consensus_queue.cc:260] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.121640 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 201) took 1531 ms. Trace:
I20260320 15:27:24.121723 28736 ts_tablet_manager.cc:1434] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:24.130014 28736 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.132139 28736 tablet_bootstrap.cc:654] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.133055 28729 ts_tablet_manager.cc:1929] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.133867 28735 tablet_replica.cc:333] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.134147 28735 raft_consensus.cc:2243] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.134449 28735 raft_consensus.cc:2272] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.135172 28735 ts_tablet_manager.cc:1916] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.135325 28729 log.cc:1199] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:24.136581 28729 ts_tablet_manager.cc:1950] T 14be8e6ed89b413a84f9517ca5ff02dc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:24.137310 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 07ff84dedc074dabb4e8e2d8460066e7 has no consensus state
I20260320 15:27:24.137842 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet f70530cd63a0420fb51527759c951e31 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.138281 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 210) took 1193 ms. Trace:
I20260320 15:27:24.138408 28729 rpcz_store.cc:276] 0320 15:27:22.944331 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.944523 (+ 192us) service_pool.cc:225] Handling call
0320 15:27:24.098960 (+1154437us) ts_tablet_manager.cc:1151] Deleting tablet 14be8e6ed89b413a84f9517ca5ff02dc
0320 15:27:24.098977 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.119726 (+ 20749us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.132847 (+ 13121us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.138208 (+ 5361us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1154064}
I20260320 15:27:24.138793 28729 tablet_replica.cc:333] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.138980 28729 raft_consensus.cc:2243] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.139173 28729 raft_consensus.cc:2272] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.139783 28729 ts_tablet_manager.cc:1916] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.141557 28734 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.141780 28734 ts_tablet_manager.cc:1403] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.069s user 0.001s sys 0.020s
I20260320 15:27:24.142720 28734 raft_consensus.cc:359] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.143014 28734 raft_consensus.cc:385] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.143126 28734 raft_consensus.cc:740] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.143455 28734 consensus_queue.cc:260] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.144250 28734 ts_tablet_manager.cc:1434] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:24.144649 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.153960 28734 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:24.156342 28734 tablet_bootstrap.cc:654] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.157226 28045 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:24.162276 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.163090 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.166183 28493 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:24.167261 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.173059 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 2165371f4b5448c7a2625926082ee4c6
I20260320 15:27:24.173708 28048 catalog_manager.cc:6741] ReplaceTablet: tablet 2165371f4b5448c7a2625926082ee4c6 deleted and replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:24.174258 28729 ts_tablet_manager.cc:1929] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.177748 28735 ts_tablet_manager.cc:1929] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.181504 28735 log.cc:1199] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1268b25a1c99404aa77e5b1bcc415f48
I20260320 15:27:24.182530 28735 ts_tablet_manager.cc:1950] T 1268b25a1c99404aa77e5b1bcc415f48 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.181543 28734 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.182812 28734 ts_tablet_manager.cc:1403] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260320 15:27:24.183746 28734 raft_consensus.cc:359] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.183951 28734 raft_consensus.cc:385] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.184056 28734 raft_consensus.cc:740] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.184300 28734 consensus_queue.cc:260] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.184949 28734 ts_tablet_manager.cc:1434] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:24.185134 28729 log.cc:1199] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:24.183503 29035 raft_consensus.cc:493] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.186249 28729 ts_tablet_manager.cc:1950] T 81a1e0957439458083e3a86ac2f01b1e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.186187 29035 raft_consensus.cc:515] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.187093 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 213) took 1138 ms. Trace:
I20260320 15:27:24.187222 28729 rpcz_store.cc:276] 0320 15:27:23.048370 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.048436 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:24.138696 (+1090260us) ts_tablet_manager.cc:1151] Deleting tablet 81a1e0957439458083e3a86ac2f01b1e
0320 15:27:24.138707 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.170672 (+ 31965us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.174101 (+ 3429us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.187052 (+ 12951us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1089929}
I20260320 15:27:24.185669 28734 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:24.188783 28240 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:24.189790 28734 tablet_bootstrap.cc:654] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:24.190042 28239 leader_election.cc:343] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:24.190228 28239 leader_election.cc:304] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:24.190618 28859 raft_consensus.cc:2749] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:24.190858 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.191390 29035 leader_election.cc:290] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.102007 28733 tablet_bootstrap.cc:654] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.193632 28730 rpcz_store.cc:276] 0320 15:27:22.590576 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.591398 (+ 822us) service_pool.cc:225] Handling call
0320 15:27:24.121421 (+1530023us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:24.121435 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.121592 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527164}
I20260320 15:27:24.194468 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 202) took 1545 ms. Trace:
I20260320 15:27:24.177395 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.194602 28730 rpcz_store.cc:276] 0320 15:27:22.649481 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.653052 (+ 3571us) service_pool.cc:225] Handling call
0320 15:27:24.194279 (+1541227us) ts_tablet_manager.cc:1151] Deleting tablet f640a6ae307140d2aee3992333e96d84
0320 15:27:24.194292 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.194423 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1534405}
I20260320 15:27:24.195088 28730 tablet_replica.cc:333] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:24.195309 28734 tablet_bootstrap.cc:492] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.195436 28730 raft_consensus.cc:2243] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.195497 28734 ts_tablet_manager.cc:1403] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260320 15:27:24.195818 28730 raft_consensus.cc:2272] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.196558 28730 ts_tablet_manager.cc:1916] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.196501 28734 raft_consensus.cc:359] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.196738 28734 raft_consensus.cc:385] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.196828 28734 raft_consensus.cc:740] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.197084 28734 consensus_queue.cc:260] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.197729 28734 ts_tablet_manager.cc:1434] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:24.198064 28734 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:24.199901 28734 tablet_bootstrap.cc:654] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.200160 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.200261 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 212) took 1206 ms. Trace:
I20260320 15:27:24.200394 28729 rpcz_store.cc:276] 0320 15:27:22.993917 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.054603 (+ 60686us) service_pool.cc:225] Handling call
0320 15:27:24.187521 (+1132918us) ts_tablet_manager.cc:1151] Deleting tablet 66a9ddea49e641da881fba339ddef549
0320 15:27:24.187562 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.200207 (+ 12645us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1132067}
I20260320 15:27:24.200810 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.200878 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 215) took 1143 ms. Trace:
I20260320 15:27:24.201083 28729 rpcz_store.cc:276] 0320 15:27:23.057427 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.057486 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:24.200716 (+1143230us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:24.200725 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.200840 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1142920}
I20260320 15:27:24.201759 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 214) took 1145 ms. Trace:
I20260320 15:27:24.201953 28729 rpcz_store.cc:276] 0320 15:27:23.056200 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.066236 (+ 10036us) service_pool.cc:225] Handling call
0320 15:27:24.201589 (+1135353us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:24.201603 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.201718 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1134809}
I20260320 15:27:24.205885 28729 tablet_replica.cc:333] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.206336 28729 raft_consensus.cc:2243] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.206678 28729 raft_consensus.cc:2272] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.207644 28729 ts_tablet_manager.cc:1916] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.216428 28733 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.217705 28735 tablet_replica.cc:333] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.223275 28733 ts_tablet_manager.cc:1403] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.243s user 0.002s sys 0.010s
I20260320 15:27:24.224406 28733 raft_consensus.cc:359] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.224885 28733 raft_consensus.cc:385] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.225023 28733 raft_consensus.cc:740] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.225432 28733 consensus_queue.cc:260] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.226653 28733 ts_tablet_manager.cc:1434] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:24.227231 28733 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.227694 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.227989 28730 ts_tablet_manager.cc:1929] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.228206 28730 log.cc:1199] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/1268b25a1c99404aa77e5b1bcc415f48
I20260320 15:27:24.228710 28730 ts_tablet_manager.cc:1950] T 1268b25a1c99404aa77e5b1bcc415f48 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:24.218416 28736 tablet_bootstrap.cc:492] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.229348 28736 ts_tablet_manager.cc:1403] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.099s user 0.007s sys 0.004s
I20260320 15:27:24.232358 28729 ts_tablet_manager.cc:1929] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.231351 28736 raft_consensus.cc:359] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.232932 28736 raft_consensus.cc:385] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.233065 28736 raft_consensus.cc:740] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.233233 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 204) took 1466 ms. Trace:
I20260320 15:27:24.233435 28736 consensus_queue.cc:260] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.234045 28736 ts_tablet_manager.cc:1434] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:24.234280 28735 raft_consensus.cc:2243] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:24.234607 28735 pending_rounds.cc:70] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:24.234676 28735 pending_rounds.cc:77] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387736829321216 op_type: NO_OP noop_request { }
I20260320 15:27:24.234800 28735 raft_consensus.cc:2889] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:24.234899 28735 raft_consensus.cc:2272] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.237942 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_874d6a667bff4c3d815fcff90cae13f5" } modify_external_catalogs: true
I20260320 15:27:24.238246 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_874d6a667bff4c3d815fcff90cae13f5" } modify_external_catalogs: true
I20260320 15:27:24.239333 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9b993c9b6ae14ae98c2239abd65333ea (table default.Table_307076d582aa44b9be757a3307398132 [id=b0945195797d43eb8b9a045e78467726]) successfully deleted
W20260320 15:27:24.239687 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 906f207dee674a4288c8c60b5d3ff9f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 906f207dee674a4288c8c60b5d3ff9f8 already in progress: creating tablet
I20260320 15:27:24.241640 28054 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:24.241921 28054 catalog_manager.cc:5671] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:24.244179 28054 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:24.246246 28729 log.cc:1199] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f640a6ae307140d2aee3992333e96d84
I20260320 15:27:24.246327 28736 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.247020 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:24.247984 28729 ts_tablet_manager.cc:1950] T f640a6ae307140d2aee3992333e96d84 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.248914 28736 tablet_bootstrap.cc:654] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:24.249044 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adb65832eefe4bd4b08d95aa83c7bd7b already in progress: creating tablet
W20260320 15:27:24.249217 28040 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 55041.2us
I20260320 15:27:24.249401 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 219) took 1102 ms. Trace:
I20260320 15:27:24.249549 28729 rpcz_store.cc:276] 0320 15:27:23.146563 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.146623 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:24.202599 (+1055976us) ts_tablet_manager.cc:1151] Deleting tablet f640a6ae307140d2aee3992333e96d84
0320 15:27:24.205776 (+ 3177us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.212613 (+ 6837us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.232124 (+ 19511us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.249315 (+ 17191us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1055716}
I20260320 15:27:24.250406 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 220) took 1102 ms. Trace:
I20260320 15:27:24.250622 28729 rpcz_store.cc:276] 0320 15:27:23.148098 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.148152 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:24.250158 (+1102006us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:24.250173 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.250328 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101783}
I20260320 15:27:24.251399 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 221) took 1102 ms. Trace:
I20260320 15:27:24.251649 28729 rpcz_store.cc:276] 0320 15:27:23.149154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.149209 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:24.251197 (+1101988us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:24.251210 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.251327 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101768}
W20260320 15:27:24.252264 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet f640a6ae307140d2aee3992333e96d84 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f640a6ae307140d2aee3992333e96d84 already in progress: creating tablet
I20260320 15:27:24.252477 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 222) took 1101 ms. Trace:
I20260320 15:27:24.252652 28729 rpcz_store.cc:276] 0320 15:27:23.150793 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.150844 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:24.252190 (+1101346us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:24.252203 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.252314 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101163}
I20260320 15:27:24.253419 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 75775842ffbb4ff1a61ba25bf6906c4b
I20260320 15:27:24.254001 28729 tablet_replica.cc:333] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.254369 28729 raft_consensus.cc:2243] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.254724 28729 raft_consensus.cc:2272] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.255738 28729 ts_tablet_manager.cc:1916] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.258380 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1268b25a1c99404aa77e5b1bcc415f48 (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:24.258675 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 14be8e6ed89b413a84f9517ca5ff02dc (table default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]) successfully deleted
I20260320 15:27:24.262804 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 81a1e0957439458083e3a86ac2f01b1e (table default.Table_e25b7a014de24f8c8f22a3584fd0d367 [id=19af05ab6c6544b99f6b348f2c52b62b]) successfully deleted
W20260320 15:27:24.263118 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 66a9ddea49e641da881fba339ddef549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a9ddea49e641da881fba339ddef549 already in progress: creating tablet
I20260320 15:27:24.249423 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7a91c52621c94498bc3d4d579569893c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.264734 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 757b5afdea674217909610e3719a8804 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 757b5afdea674217909610e3719a8804 already in progress: creating tablet
I20260320 15:27:24.265033 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet f640a6ae307140d2aee3992333e96d84 (table default.Table_d27df02667144e339d2b08cacb4b5e36 [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
W20260320 15:27:24.267066 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6201b74fdb0428e98f2eebb0b24d01a already in progress: creating tablet
W20260320 15:27:24.267330 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b2355eadcae6483bae7ae475d74dd131 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2355eadcae6483bae7ae475d74dd131 already in progress: creating tablet
I20260320 15:27:24.269066 28729 ts_tablet_manager.cc:1929] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.269317 28729 log.cc:1199] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6b6165eab6b0442fa9d94ce577c1b43c
W20260320 15:27:24.269503 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c752fb99ef494bda9d144d0e180e3100 already in progress: creating tablet
I20260320 15:27:24.270777 28729 ts_tablet_manager.cc:1950] T 6b6165eab6b0442fa9d94ce577c1b43c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.271989 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 223) took 1120 ms. Trace:
I20260320 15:27:24.272307 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.250118 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.255187 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.233656 28730 rpcz_store.cc:276] 0320 15:27:22.766627 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.766688 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:24.194966 (+1428278us) ts_tablet_manager.cc:1151] Deleting tablet 1268b25a1c99404aa77e5b1bcc415f48
0320 15:27:24.194980 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.214017 (+ 19037us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.225848 (+ 11831us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.233161 (+ 7313us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1428052}
I20260320 15:27:24.272239 28729 rpcz_store.cc:276] 0320 15:27:23.151762 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.151815 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:24.253880 (+1102065us) ts_tablet_manager.cc:1151] Deleting tablet 6b6165eab6b0442fa9d94ce577c1b43c
0320 15:27:24.253893 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.263522 (+ 9629us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.268734 (+ 5212us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.271922 (+ 3188us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1101821}
I20260320 15:27:24.262542 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 (DEFAULT_TABLE table=default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.276599 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be03035dc3d44f9aa5b43b49281a940. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:24.276703 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a91c52621c94498bc3d4d579569893c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.278883 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.279807 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3c78548647f4450687bd63dcc8123436"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.280033 28425 tablet_service.cc:1511] Processing CreateTablet for tablet aafd1e123cd043e7828140d2443e047f (DEFAULT_TABLE table=default.Table_9ee2dbc7d4f343f2b58a9fee8543896a [id=5186e4c77f9f4f59814887534793f0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.280503 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aafd1e123cd043e7828140d2443e047f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:24.282660 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c78548647f4450687bd63dcc8123436 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.284987 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.285499 28729 tablet_replica.cc:333] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.285741 28729 raft_consensus.cc:2243] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.287490 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 205) took 1425 ms. Trace:
I20260320 15:27:24.287844 28729 raft_consensus.cc:2272] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.288566 28729 ts_tablet_manager.cc:1916] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.270761 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:19 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.273484 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.290062 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 1268b25a1c99404aa77e5b1bcc415f48 (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:24.287648 28730 rpcz_store.cc:276] 0320 15:27:22.862192 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.862267 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:24.285043 (+1422776us) ts_tablet_manager.cc:1151] Deleting tablet 66a9ddea49e641da881fba339ddef549
0320 15:27:24.285056 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.287405 (+ 2349us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1422455}
I20260320 15:27:24.291205 29025 raft_consensus.cc:493] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.291291 28917 raft_consensus.cc:493] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.291440 29025 raft_consensus.cc:515] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.291580 28917 raft_consensus.cc:515] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.293851 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.294687 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d1db9963c4645ae9aab72b4b22d6938" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.295428 29025 leader_election.cc:290] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:24.296275 28383 leader_election.cc:343] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:24.298357 28383 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.298528 28917 leader_election.cc:290] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.294502 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d1db9963c4645ae9aab72b4b22d6938" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:24.273857 28735 ts_tablet_manager.cc:1916] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.294855 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:24.297680 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3f6d9a528946908891f77f951d23dd (DEFAULT_TABLE table=default.Table_83f833bca071404790ce7888026891b7 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.299697 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3f6d9a528946908891f77f951d23dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.273576 28733 tablet_bootstrap.cc:654] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.303696 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7f0242a7968541f480367b45cda5d9cc
W20260320 15:27:24.305192 28382 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.305274 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.305457 28382 leader_election.cc:304] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:24.306298 28382 leader_election.cc:343] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.306522 28382 leader_election.cc:304] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:24.297899 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 796c907919df47bf9369e45a1248f96b (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.253513 28549 tablet_service.cc:1511] Processing CreateTablet for tablet aafd1e123cd043e7828140d2443e047f (DEFAULT_TABLE table=default.Table_9ee2dbc7d4f343f2b58a9fee8543896a [id=5186e4c77f9f4f59814887534793f0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.297925 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 (DEFAULT_TABLE table=default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.308038 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 796c907919df47bf9369e45a1248f96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.308001 28917 raft_consensus.cc:2749] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:24.308496 28917 raft_consensus.cc:2749] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:24.311547 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90cd18e2ccbf48c8b804329bd3abf847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.315847 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aafd1e123cd043e7828140d2443e047f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.275203 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.278036 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.278272 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.278672 28279 tablet_service.cc:1511] Processing CreateTablet for tablet aafd1e123cd043e7828140d2443e047f (DEFAULT_TABLE table=default.Table_9ee2dbc7d4f343f2b58a9fee8543896a [id=5186e4c77f9f4f59814887534793f0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.324457 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aafd1e123cd043e7828140d2443e047f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.325438 29035 consensus_queue.cc:1048] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:24.278878 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 75775842ffbb4ff1a61ba25bf6906c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.286271 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 (DEFAULT_TABLE table=default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.328388 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 207) took 1381 ms. Trace:
I20260320 15:27:24.328524 28730 rpcz_store.cc:276] 0320 15:27:22.947302 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.948155 (+ 853us) service_pool.cc:225] Handling call
0320 15:27:24.328143 (+1379988us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:24.328158 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.328302 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1378657}
I20260320 15:27:24.328776 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be03035dc3d44f9aa5b43b49281a940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.329059 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 206) took 1383 ms. Trace:
I20260320 15:27:24.329174 28730 rpcz_store.cc:276] 0320 15:27:22.945863 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:22.946078 (+ 215us) service_pool.cc:225] Handling call
0320 15:27:24.328864 (+1382786us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:24.328874 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.328987 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1379203}
I20260320 15:27:24.329599 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 212) took 1310 ms. Trace:
I20260320 15:27:24.329703 28730 rpcz_store.cc:276] 0320 15:27:23.019342 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.019417 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:24.329417 (+1310000us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:24.329426 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.329538 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1309672}
I20260320 15:27:24.330080 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 213) took 1297 ms. Trace:
I20260320 15:27:24.330188 28730 rpcz_store.cc:276] 0320 15:27:23.032378 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.032451 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:24.329921 (+1297470us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:24.329931 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.330017 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1297155}
I20260320 15:27:24.330588 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 214) took 1294 ms. Trace:
I20260320 15:27:24.330677 28730 rpcz_store.cc:276] 0320 15:27:23.035825 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.035996 (+ 171us) service_pool.cc:225] Handling call
0320 15:27:24.330427 (+1294431us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:24.330439 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.330523 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1294117}
I20260320 15:27:24.332244 28634 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:24.332527 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.345242 28729 ts_tablet_manager.cc:1929] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.345446 28729 log.cc:1199] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:24.345873 28729 ts_tablet_manager.cc:1950] T b0e8f37a490a4fdfbc2a3dc56f6da297 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.346848 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 225) took 1181 ms. Trace:
I20260320 15:27:24.346980 28729 rpcz_store.cc:276] 0320 15:27:23.164981 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.165230 (+ 249us) service_pool.cc:225] Handling call
0320 15:27:24.285413 (+1120183us) ts_tablet_manager.cc:1151] Deleting tablet b0e8f37a490a4fdfbc2a3dc56f6da297
0320 15:27:24.285424 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.336307 (+ 50883us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.345066 (+ 8759us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.346767 (+ 1701us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1119932}
I20260320 15:27:24.348917 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 230) took 1167 ms. Trace:
I20260320 15:27:24.349133 28729 rpcz_store.cc:276] 0320 15:27:23.181520 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.183210 (+ 1690us) service_pool.cc:225] Handling call
0320 15:27:24.348677 (+1165467us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:24.348692 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.348834 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1165195}
I20260320 15:27:24.349808 28729 tablet_replica.cc:333] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.350114 28729 raft_consensus.cc:2243] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.350245 28736 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.350450 28729 raft_consensus.cc:2272] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.350488 28736 ts_tablet_manager.cc:1403] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.113s user 0.017s sys 0.021s
I20260320 15:27:24.351218 28729 ts_tablet_manager.cc:1916] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.351372 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet f70530cd63a0420fb51527759c951e31
I20260320 15:27:24.351985 28053 catalog_manager.cc:6741] ReplaceTablet: tablet f70530cd63a0420fb51527759c951e31 deleted and replaced by tablet c3bc4dd47817490d88725127b456cc70
I20260320 15:27:24.352546 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_501982d031484c3eb2382cf055a9d3be" } new_table_name: "default.Table_38da3bdd779d4eb5b0d4bb0915cedbfb" modify_external_catalogs: true
I20260320 15:27:24.352499 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a74831451d0047c2a1a67b8b8e2b93d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.353298 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a74831451d0047c2a1a67b8b8e2b93d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.356235 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_584780f08e8f4a74b5494b7c0d7ee3d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.357581 28736 raft_consensus.cc:359] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.357776 28736 raft_consensus.cc:385] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.357856 28736 raft_consensus.cc:740] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.358106 28736 consensus_queue.cc:260] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.358831 28736 ts_tablet_manager.cc:1434] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:24.359272 28736 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:24.356885 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 66a9ddea49e641da881fba339ddef549 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a9ddea49e641da881fba339ddef549 already in progress: creating tablet
I20260320 15:27:24.359628 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:24.359742 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 757b5afdea674217909610e3719a8804 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 757b5afdea674217909610e3719a8804 already in progress: creating tablet
W20260320 15:27:24.361128 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6201b74fdb0428e98f2eebb0b24d01a already in progress: creating tablet
W20260320 15:27:24.361464 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b2355eadcae6483bae7ae475d74dd131 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2355eadcae6483bae7ae475d74dd131 already in progress: creating tablet
W20260320 15:27:24.361848 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c752fb99ef494bda9d144d0e180e3100 already in progress: creating tablet
I20260320 15:27:24.362603 28736 tablet_bootstrap.cc:654] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.363590 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.365012 28563 tablet_service.cc:1511] Processing CreateTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 (DEFAULT_TABLE table=default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:24.366878 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:24.367066 28524 consensus_peers.cc:597] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6b6165eab6b0442fa9d94ce577c1b43c. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:24.374799 28736 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.375185 28736 ts_tablet_manager.cc:1403] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260320 15:27:24.376219 28736 raft_consensus.cc:359] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.377820 28736 raft_consensus.cc:385] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.377976 28736 raft_consensus.cc:740] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.378353 28736 consensus_queue.cc:260] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.379155 28736 ts_tablet_manager.cc:1434] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260320 15:27:24.379864 28736 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.381464 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:24.383829 28736 tablet_bootstrap.cc:654] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.385905 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.387001 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.387878 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.388932 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_db6589b29a164cbc95f829610dd012d0" } new_table_name: "default.Table_42f7889a62f945acac082806f1cec3f4" modify_external_catalogs: true
I20260320 15:27:24.391191 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ff78e3b872da42d8a80c4592a3f702b2" } new_table_name: "default.Table_80afab5a1d064b83b4e752b4d4c0cf61" modify_external_catalogs: true
I20260320 15:27:24.391386 28729 ts_tablet_manager.cc:1929] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.391600 28729 log.cc:1199] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ce76eeddb583441bb18fa5fb8793ffb0
I20260320 15:27:24.392140 28729 ts_tablet_manager.cc:1950] T ce76eeddb583441bb18fa5fb8793ffb0 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.394562 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 (DEFAULT_TABLE table=default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.394790 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 (DEFAULT_TABLE table=default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.395049 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7be03035dc3d44f9aa5b43b49281a940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.395412 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 796c907919df47bf9369e45a1248f96b (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.395880 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 796c907919df47bf9369e45a1248f96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.395829 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 796c907919df47bf9369e45a1248f96b (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.396193 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 796c907919df47bf9369e45a1248f96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.400017 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3f6d9a528946908891f77f951d23dd (DEFAULT_TABLE table=default.Table_83f833bca071404790ce7888026891b7 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.400580 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3f6d9a528946908891f77f951d23dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.403975 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 (DEFAULT_TABLE table=default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.404485 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90cd18e2ccbf48c8b804329bd3abf847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.405949 28550 tablet_service.cc:1511] Processing CreateTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 (DEFAULT_TABLE table=default.Table_3979e77f6aba4d9f80ba813c92287b84 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.407932 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3f6d9a528946908891f77f951d23dd (DEFAULT_TABLE table=default.Table_83f833bca071404790ce7888026891b7 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.408388 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3f6d9a528946908891f77f951d23dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.410980 29025 raft_consensus.cc:493] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.411167 29025 raft_consensus.cc:515] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.412056 29025 leader_election.cc:290] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.413113 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:24.413476 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90cd18e2ccbf48c8b804329bd3abf847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.413926 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea6ff59d2ffe4d579e6db2375eb11414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.414352 28733 tablet_bootstrap.cc:492] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:24.414510 28382 leader_election.cc:343] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.414588 28733 ts_tablet_manager.cc:1403] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.188s user 0.005s sys 0.000s
I20260320 15:27:24.415941 28733 raft_consensus.cc:359] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.416636 28733 raft_consensus.cc:385] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.417321 28733 raft_consensus.cc:740] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.417670 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.417917 28303 raft_consensus.cc:2468] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.417929 28733 consensus_queue.cc:260] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.418596 28383 leader_election.cc:304] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:24.419184 29025 raft_consensus.cc:2804] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.419282 29025 raft_consensus.cc:493] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.419409 29025 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.419428 28733 ts_tablet_manager.cc:1434] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:24.419797 28733 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.422740 28733 tablet_bootstrap.cc:654] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.422915 29025 raft_consensus.cc:515] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.425060 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.426283 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59a3c9fa17a4babaa9a18f18d51f193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.426733 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 220) took 1221 ms. Trace:
I20260320 15:27:24.431368 28730 rpcz_store.cc:276] 0320 15:27:23.204837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.204921 (+ 84us) service_pool.cc:225] Handling call
0320 15:27:24.339877 (+1134956us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:24.426490 (+ 86613us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.426663 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1084086}
I20260320 15:27:24.431921 28730 tablet_replica.cc:333] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:24.432122 28730 raft_consensus.cc:2243] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.432327 28730 raft_consensus.cc:2272] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.432564 28733 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.432716 28733 ts_tablet_manager.cc:1403] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260320 15:27:24.432899 28730 ts_tablet_manager.cc:1916] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.433622 28733 raft_consensus.cc:359] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.435714 28733 raft_consensus.cc:385] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.435704 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:24.435932 28303 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.436131 28733 raft_consensus.cc:740] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.436760 28733 consensus_queue.cc:260] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.437916 28733 ts_tablet_manager.cc:1434] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:24.438248 28733 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.438704 28303 raft_consensus.cc:2468] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:24.439986 28733 tablet_bootstrap.cc:654] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.440347 28383 leader_election.cc:304] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:24.441061 28917 raft_consensus.cc:2804] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:24.445479 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:24.446110 28733 tablet_bootstrap.cc:492] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:24.446416 28382 leader_election.cc:343] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.446538 28279 tablet_service.cc:1511] Processing CreateTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 (DEFAULT_TABLE table=default.Table_3979e77f6aba4d9f80ba813c92287b84 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.447110 28733 ts_tablet_manager.cc:1403] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260320 15:27:24.447513 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.448077 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.448477 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.448983 28274 tablet_service.cc:1511] Processing CreateTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 (DEFAULT_TABLE table=default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.448838 28733 raft_consensus.cc:359] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.449290 28733 raft_consensus.cc:385] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.449514 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.450136 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59a3c9fa17a4babaa9a18f18d51f193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.449090 28417 tablet_service.cc:1511] Processing CreateTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 (DEFAULT_TABLE table=default.Table_3979e77f6aba4d9f80ba813c92287b84 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.450631 28733 raft_consensus.cc:740] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.451006 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59a3c9fa17a4babaa9a18f18d51f193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.450938 28733 consensus_queue.cc:260] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.449591 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.451592 28730 ts_tablet_manager.cc:1929] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.449769 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.451766 28730 log.cc:1199] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ce76eeddb583441bb18fa5fb8793ffb0
I20260320 15:27:24.449942 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.450114 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.451980 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea6ff59d2ffe4d579e6db2375eb11414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.452229 28730 ts_tablet_manager.cc:1950] T ce76eeddb583441bb18fa5fb8793ffb0 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:24.453294 28733 ts_tablet_manager.cc:1434] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:24.453931 28733 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.455662 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:24.393767 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_8b29771187ba40789c95c13f432c263c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.456992 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 906f207dee674a4288c8c60b5d3ff9f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 906f207dee674a4288c8c60b5d3ff9f8 already in progress: creating tablet
I20260320 15:27:24.457532 28733 tablet_bootstrap.cc:654] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.457922 28273 tablet_service.cc:1558] Processing DeleteTablet for tablet da957aed322f46acaf5876c567c2ff94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.458606 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.459460 28444 raft_consensus.cc:3060] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.464555 29035 consensus_queue.cc:1048] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:24.465711 28736 tablet_bootstrap.cc:492] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.465988 28736 ts_tablet_manager.cc:1403] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.086s user 0.000s sys 0.010s
I20260320 15:27:24.465822 28733 tablet_bootstrap.cc:492] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.467804 28733 ts_tablet_manager.cc:1403] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260320 15:27:24.469204 28733 raft_consensus.cc:359] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.469272 28736 raft_consensus.cc:359] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.469591 28736 raft_consensus.cc:385] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.469738 28736 raft_consensus.cc:740] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.470216 28736 consensus_queue.cc:260] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.470724 28733 raft_consensus.cc:385] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.470759 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.471024 28733 raft_consensus.cc:740] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.471387 28733 consensus_queue.cc:260] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.472653 28733 ts_tablet_manager.cc:1434] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:24.473523 28733 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.474961 28736 ts_tablet_manager.cc:1434] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260320 15:27:24.475957 28733 tablet_bootstrap.cc:654] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.478878 29037 raft_consensus.cc:493] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.478986 29025 leader_election.cc:290] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.479063 29037 raft_consensus.cc:515] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.479736 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.479873 29037 leader_election.cc:290] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:24.482435 28524 leader_election.cc:343] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:24.482928 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.484143 28736 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.480875 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a1e0957439458083e3a86ac2f01b1e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.486285 28303 raft_consensus.cc:2468] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:24.487013 28524 leader_election.cc:304] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:24.487517 29037 raft_consensus.cc:2804] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.487666 29037 raft_consensus.cc:493] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.487754 29037 raft_consensus.cc:3060] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.488865 28733 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.489030 28733 ts_tablet_manager.cc:1403] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260320 15:27:24.489178 28729 tablet_replica.cc:333] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.489416 28729 raft_consensus.cc:2243] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.489594 28729 raft_consensus.cc:2272] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.489990 28733 raft_consensus.cc:359] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.490204 28733 raft_consensus.cc:385] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.490207 28729 ts_tablet_manager.cc:1916] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.490305 28733 raft_consensus.cc:740] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.490952 29037 raft_consensus.cc:515] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.492468 29037 leader_election.cc:290] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:24.492703 28733 consensus_queue.cc:260] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.493737 28733 ts_tablet_manager.cc:1434] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:24.494244 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a1e0957439458083e3a86ac2f01b1e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:24.493839 28524 leader_election.cc:343] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:24.494638 28733 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.494642 28303 raft_consensus.cc:3060] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.497983 28303 raft_consensus.cc:2468] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:24.500507 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.498332 28736 tablet_bootstrap.cc:654] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.501333 28524 leader_election.cc:304] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:24.505165 28917 raft_consensus.cc:697] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:24.514267 28917 consensus_queue.cc:237] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.532660 28733 tablet_bootstrap.cc:654] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:24.532636 28524 consensus_peers.cc:597] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:24.532917 29037 raft_consensus.cc:2804] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:24.548646 28729 ts_tablet_manager.cc:1929] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.553766 29037 raft_consensus.cc:697] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:24.554361 29037 consensus_queue.cc:237] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.556195 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:24.559137 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b29771187ba40789c95c13f432c263c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.567730 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3f64634240124b26a5e6b9f380839a17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.571202 28729 log.cc:1199] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:24.573191 28733 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.563675 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.573415 28733 ts_tablet_manager.cc:1403] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.079s user 0.000s sys 0.024s
I20260320 15:27:24.574401 28733 raft_consensus.cc:359] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.574683 28733 raft_consensus.cc:385] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.574780 28733 raft_consensus.cc:740] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.575119 28733 consensus_queue.cc:260] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.575780 28733 ts_tablet_manager.cc:1434] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:24.576117 28733 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.578382 28733 tablet_bootstrap.cc:654] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.579094 28729 ts_tablet_manager.cc:1950] T 6d1db9963c4645ae9aab72b4b22d6938 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.568050 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]) successfully deleted
W20260320 15:27:24.579931 28040 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 122686us
I20260320 15:27:24.580629 28729 tablet_replica.cc:333] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.581187 28729 raft_consensus.cc:2243] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:24.582163 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d56de00174db4e2d8b1c7215038a04aa consensus state has no leader
I20260320 15:27:24.582839 28729 raft_consensus.cc:2272] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.583813 28729 ts_tablet_manager.cc:1916] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.586555 28733 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.586745 28733 ts_tablet_manager.cc:1403] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260320 15:27:24.587342 28917 raft_consensus.cc:493] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.587669 28733 raft_consensus.cc:359] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.587883 28733 raft_consensus.cc:385] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.587980 28733 raft_consensus.cc:740] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.587692 28917 raft_consensus.cc:515] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.590337 28733 consensus_queue.cc:260] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:24.590766 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f64634240124b26a5e6b9f380839a17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.591715 28733 ts_tablet_manager.cc:1434] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:24.592077 28733 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.592247 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 (DEFAULT_TABLE table=default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.592631 28556 tablet_service.cc:1511] Processing CreateTablet for tablet fa54115d4df74bec85970057b5c3dd51 (DEFAULT_TABLE table=default.Table_26547ac0e535447fbdbc2aa66519b8fe [id=672482e082054df5a37f0eb4413aa8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.592751 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ff84dedc074dabb4e8e2d8460066e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.593039 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa54115d4df74bec85970057b5c3dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.594274 28733 tablet_bootstrap.cc:654] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.598348 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:24.599088 28383 leader_election.cc:343] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:24.600620 28917 leader_election.cc:290] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.600585 28048 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:24.583950 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6b6165eab6b0442fa9d94ce577c1b43c (table default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]) successfully deleted
I20260320 15:27:24.604264 28552 tablet_service.cc:1511] Processing CreateTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c (DEFAULT_TABLE table=default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.605875 28734 tablet_bootstrap.cc:492] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.606071 28734 ts_tablet_manager.cc:1403] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.408s user 0.112s sys 0.193s
I20260320 15:27:24.607025 28734 raft_consensus.cc:359] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.607225 28734 raft_consensus.cc:385] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.607311 28734 raft_consensus.cc:740] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.607579 28734 consensus_queue.cc:260] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.608104 28734 ts_tablet_manager.cc:1434] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:24.608337 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.608453 28734 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:24.608516 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 20a7164db1e0447fb424d8e46b747f52 (DEFAULT_TABLE table=default.Table_dd29804c0ff542cf8f2d36b963880b98 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:24.616493 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b263f9a5df2477e8dc4cd52e5834991 consensus state has no leader
I20260320 15:27:24.617007 28549 tablet_service.cc:1511] Processing CreateTablet for tablet f0a31d385b81456fb46ea52083874fcf (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.617619 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.617699 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef09e4afdd1f4fce9caec4dcafa2c50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.619199 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 224) took 1019 ms. Trace:
I20260320 15:27:24.619638 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:24.622671 28041 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 145743us
I20260320 15:27:24.623589 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:24.626286 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 906f207dee674a4288c8c60b5d3ff9f8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 906f207dee674a4288c8c60b5d3ff9f8 already in progress: creating tablet
I20260320 15:27:24.626700 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]) successfully deleted
W20260320 15:27:24.627094 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5f92561764b4cb9b15bb9a310a350f0 already in progress: creating tablet
W20260320 15:27:24.627969 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a181681cd84b7fb09c3742f4f2222f already in progress: creating tablet
I20260320 15:27:24.628624 28736 tablet_bootstrap.cc:492] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.628808 28736 ts_tablet_manager.cc:1403] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.145s user 0.017s sys 0.043s
I20260320 15:27:24.629849 28736 raft_consensus.cc:359] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.630137 28736 raft_consensus.cc:385] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.630223 28736 raft_consensus.cc:740] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.630620 28736 consensus_queue.cc:260] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:24.631724 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e9a4e61bab44d768d9adfed00c945e9 already in progress: creating tablet
W20260320 15:27:24.632014 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 583ed053b2a742c7b029f348f4495101 already in progress: creating tablet
I20260320 15:27:24.632216 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6d1db9963c4645ae9aab72b4b22d6938 (table default.Table_38da3bdd779d4eb5b0d4bb0915cedbfb [id=cc5e59bc215e444ba6c156b6d0ded84c]) successfully deleted
I20260320 15:27:24.632462 28350 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:24.632668 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.633224 28425 tablet_service.cc:1511] Processing CreateTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 (DEFAULT_TABLE table=default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.633769 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea6ff59d2ffe4d579e6db2375eb11414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.634799 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.634910 28734 tablet_bootstrap.cc:654] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.636279 28277 tablet_service.cc:1511] Processing CreateTablet for tablet fa54115d4df74bec85970057b5c3dd51 (DEFAULT_TABLE table=default.Table_26547ac0e535447fbdbc2aa66519b8fe [id=672482e082054df5a37f0eb4413aa8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.636561 28272 tablet_service.cc:1511] Processing CreateTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c (DEFAULT_TABLE table=default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.636732 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa54115d4df74bec85970057b5c3dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.637097 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 20a7164db1e0447fb424d8e46b747f52 (DEFAULT_TABLE table=default.Table_dd29804c0ff542cf8f2d36b963880b98 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.637425 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a7164db1e0447fb424d8e46b747f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.639672 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.639838 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 (DEFAULT_TABLE table=default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.640235 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ff84dedc074dabb4e8e2d8460066e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.641259 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.642232 28417 tablet_service.cc:1511] Processing CreateTablet for tablet f0a31d385b81456fb46ea52083874fcf (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.642645 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0a31d385b81456fb46ea52083874fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.644256 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef09e4afdd1f4fce9caec4dcafa2c50c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.648103 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_04b04acd60a042188e87b5d447398dce" } new_table_name: "default.Table_22b004e387544471bf99f3f2f1267d63" modify_external_catalogs: true
I20260320 15:27:24.636274 28273 tablet_service.cc:1511] Processing CreateTablet for tablet f0a31d385b81456fb46ea52083874fcf (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.648828 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0a31d385b81456fb46ea52083874fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.648734 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a861f30c768b4ee4988c1f3881eefa19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.650331 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:24.650714 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_36123e5c9fed4e51bfab33f47ed1d82b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.651178 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1a8f81dd2a33405b819cad145fe64901" } modify_external_catalogs: true
I20260320 15:27:24.651178 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.651365 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1a8f81dd2a33405b819cad145fe64901" } modify_external_catalogs: true
I20260320 15:27:24.651650 28048 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d3fccc4dd32348d1addde0067aeae899" } new_table_name: "default.Table_84e513e1494a4b71ab7f8c61376aaeae" modify_external_catalogs: true
W20260320 15:27:24.652263 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a861f30c768b4ee4988c1f3881eefa19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.653564 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_42657ecb01cc40af923263a4556db5df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.654059 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42657ecb01cc40af923263a4556db5df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:24.651243 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36123e5c9fed4e51bfab33f47ed1d82b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.651320 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2827438ad3b74f42968ceb52cf633bff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.658135 28415 tablet_service.cc:1511] Processing CreateTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c (DEFAULT_TABLE table=default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.658718 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef09e4afdd1f4fce9caec4dcafa2c50c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:24.659068 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2827438ad3b74f42968ceb52cf633bff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.667368 28584 raft_consensus.cc:2468] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.671640 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a7164db1e0447fb424d8e46b747f52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:24.674849 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ba738a76a424c18b36985a1977bcee4 consensus state has no leader
I20260320 15:27:24.675379 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3618306c0eff4c74935789df59e679f4" } modify_external_catalogs: true
I20260320 15:27:24.675748 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3618306c0eff4c74935789df59e679f4" } modify_external_catalogs: true
I20260320 15:27:24.677752 29094 raft_consensus.cc:493] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.679318 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0a31d385b81456fb46ea52083874fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.680275 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 45a0e8e2ce7a454ebcab4e59a4de68dd from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.680337 28049 master_service.cc:954] ReplaceTablet: received request to replace tablet 6d1db9963c4645ae9aab72b4b22d6938 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:24.680932 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3796f2b714f64f18a0db9057941513ff" } modify_external_catalogs: true
I20260320 15:27:24.681090 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3796f2b714f64f18a0db9057941513ff" } modify_external_catalogs: true
I20260320 15:27:24.681048 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_93591206ed3f475cb1d454da2ea159a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.681632 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ca5e0c215f574b8aafec8d5e2da9a399 consensus state has no leader
W20260320 15:27:24.681635 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93591206ed3f475cb1d454da2ea159a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.682024 28054 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_30015eb07aba409bb03a37140eea20d6" } modify_external_catalogs: true
I20260320 15:27:24.682233 28054 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_30015eb07aba409bb03a37140eea20d6" } modify_external_catalogs: true
I20260320 15:27:24.677939 29094 raft_consensus.cc:515] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.686650 28736 ts_tablet_manager.cc:1434] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.057s user 0.001s sys 0.002s
I20260320 15:27:24.687625 28736 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.689030 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 20a7164db1e0447fb424d8e46b747f52 (DEFAULT_TABLE table=default.Table_dd29804c0ff542cf8f2d36b963880b98 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.689585 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20a7164db1e0447fb424d8e46b747f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.690284 28382 leader_election.cc:304] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:24.693534 28736 tablet_bootstrap.cc:654] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.694454 28733 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:24.694353 28418 tablet_service.cc:1511] Processing CreateTablet for tablet fa54115d4df74bec85970057b5c3dd51 (DEFAULT_TABLE table=default.Table_26547ac0e535447fbdbc2aa66519b8fe [id=672482e082054df5a37f0eb4413aa8fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.699287 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet 545c8fd2d4c845adba6dbf42dfddba3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.699589 28735 ts_tablet_manager.cc:1929] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:24.699920 28735 log.cc:1199] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ce76eeddb583441bb18fa5fb8793ffb0
I20260320 15:27:24.700809 29094 leader_election.cc:290] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:24.700930 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 (DEFAULT_TABLE table=default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.678871 28730 rpcz_store.cc:276] 0320 15:27:23.600416 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.600587 (+ 171us) service_pool.cc:225] Handling call
0320 15:27:24.618989 (+1018402us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:24.619005 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.619139 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":990935}
I20260320 15:27:24.701279 28733 ts_tablet_manager.cc:1403] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.103s user 0.003s sys 0.008s
I20260320 15:27:24.701383 29086 raft_consensus.cc:2804] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.701489 29086 raft_consensus.cc:493] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.701610 29086 raft_consensus.cc:3060] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.701802 28735 ts_tablet_manager.cc:1950] T ce76eeddb583441bb18fa5fb8793ffb0 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:24.702627 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5f92561764b4cb9b15bb9a310a350f0 already in progress: creating tablet
W20260320 15:27:24.704523 28524 leader_election.cc:343] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 545c8fd2d4c845adba6dbf42dfddba3e
I20260320 15:27:24.707211 28917 raft_consensus.cc:493] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.707508 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 233) took 1188 ms. Trace:
I20260320 15:27:24.709192 28733 raft_consensus.cc:359] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.709815 28735 rpcz_store.cc:276] 0320 15:27:23.513809 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.513865 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:24.217581 (+703716us) ts_tablet_manager.cc:1151] Deleting tablet ce76eeddb583441bb18fa5fb8793ffb0
0320 15:27:24.217592 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.425655 (+208063us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.696979 (+271324us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:24.702099 (+ 5120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":684303}
I20260320 15:27:24.710559 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ce76eeddb583441bb18fa5fb8793ffb0 (table default.Table_f325d0702e60408fb5a7d3a21290c416 [id=f0e9c9ad3d504860a14df78e47af5138]) successfully deleted
I20260320 15:27:24.712143 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 234) took 1129 ms. Trace:
I20260320 15:27:24.712273 28735 rpcz_store.cc:276] 0320 15:27:23.581332 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.582105 (+ 773us) service_pool.cc:225] Handling call
0320 15:27:24.710416 (+1128311us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:24.710429 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:24.710581 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1127941}
W20260320 15:27:24.712805 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5f92561764b4cb9b15bb9a310a350f0 already in progress: creating tablet
W20260320 15:27:24.714262 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a181681cd84b7fb09c3742f4f2222f already in progress: creating tablet
W20260320 15:27:24.717044 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e9a4e61bab44d768d9adfed00c945e9 already in progress: creating tablet
W20260320 15:27:24.720096 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66a181681cd84b7fb09c3742f4f2222f already in progress: creating tablet
W20260320 15:27:24.721194 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 583ed053b2a742c7b029f348f4495101 already in progress: creating tablet
I20260320 15:27:24.724793 28735 tablet_replica.cc:333] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.725167 28735 raft_consensus.cc:2243] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:24.725203 29105 tablet.cc:1723] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:24.725670 28735 pending_rounds.cc:70] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:24.725809 28735 pending_rounds.cc:77] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387738847105024 op_type: NO_OP noop_request { }
I20260320 15:27:24.726011 28735 raft_consensus.cc:2889] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:24.726317 28735 raft_consensus.cc:2272] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.727658 28735 ts_tablet_manager.cc:1916] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.731240 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545c8fd2d4c845adba6dbf42dfddba3e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:24.735280 28444 raft_consensus.cc:2468] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:24.735594 28733 raft_consensus.cc:385] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.735986 28917 raft_consensus.cc:515] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.736194 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.738649 28736 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.738834 28736 ts_tablet_manager.cc:1403] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.002s
I20260320 15:27:24.738876 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa54115d4df74bec85970057b5c3dd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.739650 28524 leader_election.cc:304] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:24.739717 28736 raft_consensus.cc:359] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.739904 28736 raft_consensus.cc:385] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.740006 28736 raft_consensus.cc:740] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.740435 29094 raft_consensus.cc:2804] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.740345 28736 consensus_queue.cc:260] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.740971 28729 ts_tablet_manager.cc:1929] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.741015 28736 ts_tablet_manager.cc:1434] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:24.741621 28736 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.741825 28733 raft_consensus.cc:740] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.742516 29094 raft_consensus.cc:493] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.742653 29094 raft_consensus.cc:3060] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.742549 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e822249154a1784ebd66f8e56abdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.742812 28303 raft_consensus.cc:2468] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.743935 28736 tablet_bootstrap.cc:654] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.750404 29094 raft_consensus.cc:515] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.750893 28730 tablet_replica.cc:333] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:24.750947 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e822249154a1784ebd66f8e56abdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:24.755148 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e9a4e61bab44d768d9adfed00c945e9 already in progress: creating tablet
W20260320 15:27:24.756029 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 583ed053b2a742c7b029f348f4495101 already in progress: creating tablet
I20260320 15:27:24.764168 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet edb7b53bb7444a39b471218c6039542f
I20260320 15:27:24.765283 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.766594 28735 ts_tablet_manager.cc:1929] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:24.766762 28735 log.cc:1199] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/14be8e6ed89b413a84f9517ca5ff02dc
I20260320 15:27:24.767241 28735 ts_tablet_manager.cc:1950] T 14be8e6ed89b413a84f9517ca5ff02dc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.768606 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 14be8e6ed89b413a84f9517ca5ff02dc (table default.Table_57247cf8e07b450984f71c47c56e7b1b [id=19ca5fa9908a478d9a5e95ef6e017012]) successfully deleted
I20260320 15:27:24.771378 28735 tablet_replica.cc:333] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.772054 28735 raft_consensus.cc:2243] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.772305 28735 raft_consensus.cc:2272] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.772413 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.773821 29035 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.774015 29035 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.774363 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ff84dedc074dabb4e8e2d8460066e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.775270 28588 raft_consensus.cc:2468] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.775942 28729 log.cc:1199] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/28c1dcc311a14b33a289ae2b8a485d2b
I20260320 15:27:24.776187 29035 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.776355 28917 leader_election.cc:290] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.776610 28383 leader_election.cc:304] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:24.777480 28736 tablet_bootstrap.cc:492] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.777670 28736 ts_tablet_manager.cc:1403] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.003s
I20260320 15:27:24.778577 28736 raft_consensus.cc:359] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.771373 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.779071 28736 raft_consensus.cc:385] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.779165 28736 raft_consensus.cc:740] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.779500 28736 consensus_queue.cc:260] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.780289 28736 ts_tablet_manager.cc:1434] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:24.780423 28735 ts_tablet_manager.cc:1916] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:24.781708 28048 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ce8595157679444b959b569dbb7d8df0 consensus state has no leader
I20260320 15:27:24.786003 28730 raft_consensus.cc:2243] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:24.786463 28730 pending_rounds.cc:70] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:24.786784 28588 raft_consensus.cc:1275] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:24.786998 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 26818e620ff649d18b671e4f4c7c7e74 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:24.787516 28239 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:24.787590 29025 consensus_queue.cc:1048] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:24.791066 28733 consensus_queue.cc:260] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.791844 28733 ts_tablet_manager.cc:1434] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.090s user 0.000s sys 0.002s
I20260320 15:27:24.792212 28733 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:24.793787 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 81a1e0957439458083e3a86ac2f01b1e with delete_type TABLET_DATA_DELETED (replaced by tablet 931f6f88a1d94a40b69f07216b8313c8) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.794095 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.794291 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.794588 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.797468 28736 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.800120 28736 tablet_bootstrap.cc:654] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.800751 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet f640a6ae307140d2aee3992333e96d84 with delete_type TABLET_DATA_DELETED (replaced by tablet 7feb8477ebf841a19977a434f365f537) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:24.801110 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 203c619513a146f49a4825962f169f45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:24.802814 28240 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:24.803007 28240 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:24.803041 28524 leader_election.cc:343] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 545c8fd2d4c845adba6dbf42dfddba3e
I20260320 15:27:24.803479 29035 raft_consensus.cc:2749] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:24.803570 28917 raft_consensus.cc:493] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.805616 28417 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:24.806602 29025 raft_consensus.cc:2804] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.806722 29025 raft_consensus.cc:493] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.807127 29025 raft_consensus.cc:3060] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.807683 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "545c8fd2d4c845adba6dbf42dfddba3e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:24.808303 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 44750.9us
I20260320 15:27:24.810570 28303 raft_consensus.cc:3060] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.811690 28735 ts_tablet_manager.cc:1929] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.811849 28735 log.cc:1199] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2069aa42660947d8a1d37d6d40846991
I20260320 15:27:24.812314 28735 ts_tablet_manager.cc:1950] T 2069aa42660947d8a1d37d6d40846991 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.814318 28303 raft_consensus.cc:1275] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260320 15:27:24.815618 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 2069aa42660947d8a1d37d6d40846991 (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:24.816793 28736 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.816979 28736 ts_tablet_manager.cc:1403] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20260320 15:27:24.817112 28917 raft_consensus.cc:515] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.817868 28736 raft_consensus.cc:359] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.818048 28736 raft_consensus.cc:385] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.818163 28736 raft_consensus.cc:740] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.818455 28736 consensus_queue.cc:260] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.819190 28736 ts_tablet_manager.cc:1434] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:24.819561 28736 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.821497 28736 tablet_bootstrap.cc:654] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.822753 28735 tablet_replica.cc:333] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.822994 28735 raft_consensus.cc:2243] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.824653 28444 raft_consensus.cc:3060] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.825631 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da957aed322f46acaf5876c567c2ff94" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:24.826340 28300 raft_consensus.cc:2468] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.823180 28735 raft_consensus.cc:2272] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.827418 28735 ts_tablet_manager.cc:1916] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.828222 28729 ts_tablet_manager.cc:1950] T 28c1dcc311a14b33a289ae2b8a485d2b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:24.829944 28729 tablet_replica.cc:333] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:24.830816 29084 consensus_queue.cc:1048] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260320 15:27:24.832401 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 28c1dcc311a14b33a289ae2b8a485d2b (table default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]) successfully deleted
W20260320 15:27:24.832823 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2165371f4b5448c7a2625926082ee4c6 already in progress: creating tablet
W20260320 15:27:24.833139 28038 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet edb7b53bb7444a39b471218c6039542f (table default.Table_1a8f81dd2a33405b819cad145fe64901 [id=7b5c27e0814148848d52b88c9dce9553], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet edb7b53bb7444a39b471218c6039542f consensus state has no leader
I20260320 15:27:24.836542 28736 tablet_bootstrap.cc:492] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.836742 28736 ts_tablet_manager.cc:1403] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.004s
I20260320 15:27:24.837548 28736 raft_consensus.cc:359] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.837764 28736 raft_consensus.cc:385] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.837836 28736 raft_consensus.cc:740] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.837919 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.838017 29086 raft_consensus.cc:515] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.838135 28736 consensus_queue.cc:260] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.838904 28736 ts_tablet_manager.cc:1434] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260320 15:27:24.841113 28736 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.843182 28734 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:24.844458 28736 tablet_bootstrap.cc:654] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.845428 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_20ceb59162bc456f86478ad67ada6956" } modify_external_catalogs: true
I20260320 15:27:24.845674 28735 ts_tablet_manager.cc:1929] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.845711 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_20ceb59162bc456f86478ad67ada6956" } modify_external_catalogs: true
I20260320 15:27:24.845850 28735 log.cc:1199] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/97811e288242436c8e74fd9f45d8d81e
I20260320 15:27:24.846287 28735 ts_tablet_manager.cc:1950] T 97811e288242436c8e74fd9f45d8d81e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.846432 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_38da3bdd779d4eb5b0d4bb0915cedbfb" } new_table_name: "default.Table_7b1d2897e6684a89bb2fe8570e1868d2" modify_external_catalogs: true
I20260320 15:27:24.847236 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 97811e288242436c8e74fd9f45d8d81e (table default.Table_b85eb77d36ad4cd58e04e60d9db9831c [id=a98acd9e31194e47a86049e745618e73]) successfully deleted
I20260320 15:27:24.847859 28735 tablet_replica.cc:333] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.848068 28735 raft_consensus.cc:2243] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.848281 28735 raft_consensus.cc:2272] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.848323 28917 leader_election.cc:290] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.848918 28735 ts_tablet_manager.cc:1916] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.848552 28383 leader_election.cc:304] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:24.858404 28736 tablet_bootstrap.cc:492] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.858633 28736 ts_tablet_manager.cc:1403] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260320 15:27:24.859678 28736 raft_consensus.cc:359] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.859897 28736 raft_consensus.cc:385] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.859998 28736 raft_consensus.cc:740] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.860291 28736 consensus_queue.cc:260] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.861106 28736 ts_tablet_manager.cc:1434] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:24.861511 28736 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.863512 28736 tablet_bootstrap.cc:654] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.867733 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:24.870220 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f7415957b95743768b23acf4577d7db3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.871109 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7415957b95743768b23acf4577d7db3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.878983 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 54a372bb37de43938f0420bdc9c7249c (DEFAULT_TABLE table=default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 [id=6ea6362e6d934bfe8cb54c912b4932dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.879616 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a372bb37de43938f0420bdc9c7249c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.882141 29094 leader_election.cc:290] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:24.882520 28274 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c7dc6859048ce956927201df44a86 (DEFAULT_TABLE table=default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.882823 28734 ts_tablet_manager.cc:1403] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.235s user 0.010s sys 0.001s
I20260320 15:27:24.883047 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c7dc6859048ce956927201df44a86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.884078 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1bbba63ee33c407f8d02e8f63df7245e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.884197 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
W20260320 15:27:24.884733 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bbba63ee33c407f8d02e8f63df7245e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.885383 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_82d375c404eb4fdca024e9fc85f87e1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.886073 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82d375c404eb4fdca024e9fc85f87e1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.887094 29086 leader_election.cc:290] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:24.887857 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d56de00174db4e2d8b1c7215038a04aa consensus state has no leader
I20260320 15:27:24.888350 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ac4367958ed24d109d328c0c2d1098aa" } new_table_name: "default.Table_6d5fc724ebb540cbb35c767edfe19833" modify_external_catalogs: true
I20260320 15:27:24.889791 28730 pending_rounds.cc:77] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387736402214912 op_type: NO_OP noop_request { }
I20260320 15:27:24.890733 28917 raft_consensus.cc:2804] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:24.890858 28917 raft_consensus.cc:493] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:24.891556 29035 raft_consensus.cc:493] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.891731 29035 raft_consensus.cc:515] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.892382 28048 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:24.892545 29035 leader_election.cc:290] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.893209 28917 raft_consensus.cc:3060] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.893541 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_bd527e7856bf48069d98d45db0adac7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.894196 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd527e7856bf48069d98d45db0adac7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:24.894551 28240 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:24.894615 28735 ts_tablet_manager.cc:1929] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.895077 28735 log.cc:1199] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ae4ec6ec99df4fb5b1ebec5b1437c8fd
I20260320 15:27:24.895637 28735 ts_tablet_manager.cc:1950] T ae4ec6ec99df4fb5b1ebec5b1437c8fd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.896019 28734 raft_consensus.cc:359] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.897341 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:24.897611 28587 raft_consensus.cc:3060] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.897400 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da957aed322f46acaf5876c567c2ff94" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:24.898216 28586 raft_consensus.cc:2468] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:24.900177 28444 raft_consensus.cc:2468] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:24.905474 28730 raft_consensus.cc:2889] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:24.904086 29025 raft_consensus.cc:515] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.912730 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf5e17dea5445d9909a229fa8315348 (DEFAULT_TABLE table=default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.914119 28425 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c7dc6859048ce956927201df44a86 (DEFAULT_TABLE table=default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:24.914433 28383 leader_election.cc:343] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:24.914579 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 45929.4us
I20260320 15:27:24.915932 28734 raft_consensus.cc:385] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.916072 28734 raft_consensus.cc:740] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.916355 28734 consensus_queue.cc:260] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.916745 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf5e17dea5445d9909a229fa8315348 (DEFAULT_TABLE table=default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.917121 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ae4ec6ec99df4fb5b1ebec5b1437c8fd (table default.Table_856a539607904111b591f60c46495372 [id=e15e4d0b1d844bf292a75c68c77ed832]) successfully deleted
I20260320 15:27:24.917228 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf5e17dea5445d9909a229fa8315348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.918298 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf5e17dea5445d9909a229fa8315348. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:24.918891 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2165371f4b5448c7a2625926082ee4c6 already in progress: creating tablet
I20260320 15:27:24.919270 28735 tablet_replica.cc:333] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:24.919570 28735 raft_consensus.cc:2243] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:24.919800 28735 raft_consensus.cc:2272] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.920565 28735 ts_tablet_manager.cc:1916] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.920794 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e822249154a1784ebd66f8e56abdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:24.921021 28300 raft_consensus.cc:3060] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:24.924624 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 54a372bb37de43938f0420bdc9c7249c (DEFAULT_TABLE table=default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 [id=6ea6362e6d934bfe8cb54c912b4932dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.925333 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a372bb37de43938f0420bdc9c7249c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.925510 28300 raft_consensus.cc:2468] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:24.928159 28736 tablet_bootstrap.cc:492] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.928377 28736 ts_tablet_manager.cc:1403] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.007s
I20260320 15:27:24.932484 28383 leader_election.cc:304] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:24.932545 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 533105ad6e054fe3a4db32855bc688bd (DEFAULT_TABLE table=default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.933091 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533105ad6e054fe3a4db32855bc688bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.936652 28274 tablet_service.cc:1511] Processing CreateTablet for tablet ffe278739bbc4ddab848af9c25241380 (DEFAULT_TABLE table=default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.937177 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe278739bbc4ddab848af9c25241380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.940160 28273 tablet_service.cc:1511] Processing CreateTablet for tablet fcff253b9cce45b4ad11dc38397afee2 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.940685 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcff253b9cce45b4ad11dc38397afee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.941797 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c7dc6859048ce956927201df44a86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.941907 28735 ts_tablet_manager.cc:1929] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:24.942049 28735 log.cc:1199] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:24.943934 29025 leader_election.cc:290] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:24.944593 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 685e581b3f084d4a9aa97fba0a7767c7 (DEFAULT_TABLE table=default.Table_4068b4f067d74effb3bc0dc5bb1faddc [id=efaa41b4075b4bc98bf0b19fe0c05ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.945032 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685e581b3f084d4a9aa97fba0a7767c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.945900 28050 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:24.946636 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7c4b53fb135741efad5d3e59a6d4eb3e
I20260320 15:27:24.947615 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_d27df02667144e339d2b08cacb4b5e36" } new_table_name: "default.Table_fdd90d848d4846dcb37f6b50bce8156b" modify_external_catalogs: true
I20260320 15:27:24.947862 28736 raft_consensus.cc:359] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.948073 28736 raft_consensus.cc:385] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.948151 28736 raft_consensus.cc:740] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.948151 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.948608 28736 consensus_queue.cc:260] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.948817 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.949272 29086 raft_consensus.cc:2804] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:24.950012 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:24.950564 28053 catalog_manager.cc:6741] ReplaceTablet: tablet 26818e620ff649d18b671e4f4c7c7e74 deleted and replaced by tablet 8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:24.950927 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5803227c57ec4896947ecce23135454f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.951599 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5803227c57ec4896947ecce23135454f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.951838 28413 tablet_service.cc:1511] Processing CreateTablet for tablet fcff253b9cce45b4ad11dc38397afee2 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.952901 28736 ts_tablet_manager.cc:1434] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260320 15:27:24.952847 28427 tablet_service.cc:1511] Processing CreateTablet for tablet ffe278739bbc4ddab848af9c25241380 (DEFAULT_TABLE table=default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.953480 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 685e581b3f084d4a9aa97fba0a7767c7 (DEFAULT_TABLE table=default.Table_4068b4f067d74effb3bc0dc5bb1faddc [id=efaa41b4075b4bc98bf0b19fe0c05ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.953975 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.954582 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:24.955494 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_089d0cb58bd1495cbe761dae3b90b562"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:24.955632 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:24.956259 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_089d0cb58bd1495cbe761dae3b90b562 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.956970 28736 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:24.958223 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:24.958890 28736 tablet_bootstrap.cc:654] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:24.959074 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_b91605000af04532ba5d3fc895d8dc1a" } new_table_name: "default.Table_003b2dd0dbb34453bba65abed2b789e9" modify_external_catalogs: true
I20260320 15:27:24.960589 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.960637 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.967590 28735 ts_tablet_manager.cc:1950] T 81a1e0957439458083e3a86ac2f01b1e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:24.969059 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 6b6165eab6b0442fa9d94ce577c1b43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.969199 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c1dcc311a14b33a289ae2b8a485d2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.969449 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.969558 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.969945 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.970937 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:24.976928 28730 raft_consensus.cc:2272] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:24.977658 28730 ts_tablet_manager.cc:1916] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:24.978736 28054 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ecaa08ba34514c5e99a9e2d4ba9268a9
I20260320 15:27:24.979610 29094 raft_consensus.cc:493] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:24.979796 29094 raft_consensus.cc:515] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.980525 29094 leader_election.cc:290] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:24.981056 28736 tablet_bootstrap.cc:492] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:24.981218 28736 ts_tablet_manager.cc:1403] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260320 15:27:24.981420 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 54a372bb37de43938f0420bdc9c7249c (DEFAULT_TABLE table=default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 [id=6ea6362e6d934bfe8cb54c912b4932dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.982144 28736 raft_consensus.cc:359] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.982347 28736 raft_consensus.cc:385] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:24.982438 28736 raft_consensus.cc:740] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:24.982686 28736 consensus_queue.cc:260] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:24.983489 28736 ts_tablet_manager.cc:1434] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:24.984515 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:24.985258 28549 tablet_service.cc:1511] Processing CreateTablet for tablet ac4c7dc6859048ce956927201df44a86 (DEFAULT_TABLE table=default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.985966 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 1bf5e17dea5445d9909a229fa8315348 (DEFAULT_TABLE table=default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:24.986563 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54a372bb37de43938f0420bdc9c7249c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.989604 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a202f09653ee4487a0f5056420b9a8e1" } new_table_name: "default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096" modify_external_catalogs: true
I20260320 15:27:24.990221 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf5e17dea5445d9909a229fa8315348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:24.993108 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_15707876735c420fb2d7fbc028f9b1d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:24.993868 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15707876735c420fb2d7fbc028f9b1d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:24.996207 28524 leader_election.cc:304] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:24.998063 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a8e822249154a1784ebd66f8e56abdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:24.998323 28586 raft_consensus.cc:3060] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.000365 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a5952001c570473c902abb9d529c60f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.001725 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5952001c570473c902abb9d529c60f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.002007 28586 raft_consensus.cc:2468] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:25.013060 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 533105ad6e054fe3a4db32855bc688bd (DEFAULT_TABLE table=default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.013959 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac4c7dc6859048ce956927201df44a86. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:25.017961 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 105503us
I20260320 15:27:25.019117 29094 raft_consensus.cc:2804] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.021196 28552 tablet_service.cc:1511] Processing CreateTablet for tablet ffe278739bbc4ddab848af9c25241380 (DEFAULT_TABLE table=default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.022683 29094 raft_consensus.cc:697] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:25.023198 29094 consensus_queue.cc:237] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.024603 28272 tablet_service.cc:1511] Processing CreateTablet for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13 (DEFAULT_TABLE table=default.Table_a74831451d0047c2a1a67b8b8e2b93d0 [id=ada35f69f6904e7b8d50889e605f0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.027077 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.027751 28562 tablet_service.cc:1511] Processing CreateTablet for tablet fcff253b9cce45b4ad11dc38397afee2 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.029371 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533105ad6e054fe3a4db32855bc688bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.030736 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.031838 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.033496 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.033867 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.035460 28564 tablet_service.cc:1511] Processing CreateTablet for tablet f05205970cd249b68d138b904ba61c9e (DEFAULT_TABLE table=default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 [id=2d0fe40191c9403ab71159ef1377e21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.039640 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_78dd3a47806c44f186bb967be20e8d9f" } modify_external_catalogs: true
I20260320 15:27:25.041000 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:25.040730 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_78dd3a47806c44f186bb967be20e8d9f" } modify_external_catalogs: true
I20260320 15:27:25.041595 28051 catalog_manager.cc:6741] ReplaceTablet: tablet 45a0e8e2ce7a454ebcab4e59a4de68dd deleted and replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633
I20260320 15:27:25.043587 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd with delete_type TABLET_DATA_DELETED (replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.046576 28733 tablet_bootstrap.cc:654] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.046809 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 685e581b3f084d4a9aa97fba0a7767c7 (DEFAULT_TABLE table=default.Table_4068b4f067d74effb3bc0dc5bb1faddc [id=efaa41b4075b4bc98bf0b19fe0c05ba9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.047684 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c2a40da69e9d4735b3dd5e71d419b00b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.048468 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2a40da69e9d4735b3dd5e71d419b00b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:25.061641 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7feb8477ebf841a19977a434f365f537 consensus state has no leader
I20260320 15:27:25.061980 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcff253b9cce45b4ad11dc38397afee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.065702 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.066344 28284 tablet_service.cc:1511] Processing CreateTablet for tablet f05205970cd249b68d138b904ba61c9e (DEFAULT_TABLE table=default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 [id=2d0fe40191c9403ab71159ef1377e21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.069365 29086 raft_consensus.cc:697] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:25.070951 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe278739bbc4ddab848af9c25241380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.071645 28280 tablet_service.cc:1511] Processing CreateTablet for tablet c3bc4dd47817490d88725127b456cc70 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.074975 29105 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:25.074746 29086 consensus_queue.cc:237] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.077960 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.079185 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet b0e8f37a490a4fdfbc2a3dc56f6da297 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bc3fb47136d4b88a2898af399d74a8a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.079725 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e8f37a490a4fdfbc2a3dc56f6da297" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:25.080024 28303 raft_consensus.cc:2468] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:25.081789 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.082906 28735 tablet_replica.cc:333] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:25.083146 28735 raft_consensus.cc:2243] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:25.083428 28735 raft_consensus.cc:2272] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.084030 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 81a1e0957439458083e3a86ac2f01b1e (table default.Table_e25b7a014de24f8c8f22a3584fd0d367 [id=19af05ab6c6544b99f6b348f2c52b62b]) successfully deleted
W20260320 15:27:25.084379 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43a0fadc7dac4e21ad034e4cd2d70826 already in progress: creating tablet
I20260320 15:27:25.087225 28735 ts_tablet_manager.cc:1916] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.087473 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.088099 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.091604 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685e581b3f084d4a9aa97fba0a7767c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.094118 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.098520 28917 raft_consensus.cc:515] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.100641 28736 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.101109 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.102519 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d1db9963c4645ae9aab72b4b22d6938" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:25.103183 28736 tablet_bootstrap.cc:654] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.104249 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcff253b9cce45b4ad11dc38397afee2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.110265 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05205970cd249b68d138b904ba61c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.111799 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bc4dd47817490d88725127b456cc70. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:25.115140 28239 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.115327 28239 leader_election.cc:304] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.116231 29035 raft_consensus.cc:2749] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.117161 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da957aed322f46acaf5876c567c2ff94" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:25.117235 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da957aed322f46acaf5876c567c2ff94" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:25.117399 28300 raft_consensus.cc:3060] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.118422 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6462b376d8f34deba33a10a982b4fed4" } new_table_name: "default.Table_a5ca3ae6cd61430db4e2c4112c011e95" modify_external_catalogs: true
I20260320 15:27:25.120304 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_314b2abee493499f82fb7741c3a20f4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.121297 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_314b2abee493499f82fb7741c3a20f4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.125351 28551 tablet_service.cc:1511] Processing CreateTablet for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13 (DEFAULT_TABLE table=default.Table_a74831451d0047c2a1a67b8b8e2b93d0 [id=ada35f69f6904e7b8d50889e605f0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.125455 28524 leader_election.cc:304] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.126263 28560 tablet_service.cc:1511] Processing CreateTablet for tablet c3bc4dd47817490d88725127b456cc70 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.126946 28300 raft_consensus.cc:2468] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:25.129738 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd with delete_type TABLET_DATA_DELETED (replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.129878 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.130453 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e31e11048b494ebe1fd4d89488b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.130599 29037 raft_consensus.cc:2804] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:25.132287 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 (DEFAULT_TABLE table=default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.132507 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.132669 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.132851 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685e581b3f084d4a9aa97fba0a7767c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.133062 28917 leader_election.cc:290] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.134400 29107 tablet.cc:1723] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:25.135201 29037 raft_consensus.cc:493] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:25.135357 29037 raft_consensus.cc:3060] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.135507 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05205970cd249b68d138b904ba61c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.138645 29037 raft_consensus.cc:515] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.139892 29037 leader_election.cc:290] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:25.140076 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4cad900411e4c9686701aa99206dd23 consensus state has no leader
I20260320 15:27:25.140765 28588 raft_consensus.cc:3060] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.141781 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.142768 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 (DEFAULT_TABLE table=default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.143600 28735 ts_tablet_manager.cc:1929] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:25.143743 28735 log.cc:1199] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/75775842ffbb4ff1a61ba25bf6906c4b
I20260320 15:27:25.145292 28735 ts_tablet_manager.cc:1950] T 75775842ffbb4ff1a61ba25bf6906c4b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:25.145790 29107 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:25.146595 28045 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a202f09653ee4487a0f5056420b9a8e1 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 2 got 0
I20260320 15:27:25.147313 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe278739bbc4ddab848af9c25241380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.148415 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bc4dd47817490d88725127b456cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.148748 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.149844 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa5f6fe801b4452bf294a9ae23bb163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.151243 28422 tablet_service.cc:1511] Processing CreateTablet for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13 (DEFAULT_TABLE table=default.Table_a74831451d0047c2a1a67b8b8e2b93d0 [id=ada35f69f6904e7b8d50889e605f0070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.151819 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.146442 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa5f6fe801b4452bf294a9ae23bb163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.146881 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0e8f37a490a4fdfbc2a3dc56f6da297" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:25.152486 28300 raft_consensus.cc:3060] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.156100 28300 raft_consensus.cc:2468] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:25.109061 28424 tablet_service.cc:1511] Processing CreateTablet for tablet f05205970cd249b68d138b904ba61c9e (DEFAULT_TABLE table=default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 [id=2d0fe40191c9403ab71159ef1377e21c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.162405 28383 leader_election.cc:304] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:25.173470 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet da860b779a174adbb4fd6f7f7520348c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.176146 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_764f1d7877d24d50b17c6d27527fdd6d" } modify_external_catalogs: true
I20260320 15:27:25.176534 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_764f1d7877d24d50b17c6d27527fdd6d" } modify_external_catalogs: true
I20260320 15:27:25.179713 28045 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 3 got 2
I20260320 15:27:25.180912 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bb5eb65d708a4ece947d11468e68b8d4" } modify_external_catalogs: true
I20260320 15:27:25.181154 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bb5eb65d708a4ece947d11468e68b8d4" } modify_external_catalogs: true
I20260320 15:27:25.182873 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3f64634240124b26a5e6b9f380839a17" } modify_external_catalogs: true
I20260320 15:27:25.183077 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3f64634240124b26a5e6b9f380839a17" } modify_external_catalogs: true
I20260320 15:27:25.186070 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]) successfully deleted
W20260320 15:27:25.186955 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet f70530cd63a0420fb51527759c951e31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f70530cd63a0420fb51527759c951e31 already in progress: creating tablet
W20260320 15:27:25.187819 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7f0242a7968541f480367b45cda5d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f0242a7968541f480367b45cda5d9cc already in progress: creating tablet
I20260320 15:27:25.187773 28524 leader_election.cc:304] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
W20260320 15:27:25.188642 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a072859f39c4c28ae65b0e9afa79d66 already in progress: creating tablet
I20260320 15:27:25.188969 28735 tablet_replica.cc:333] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:25.189239 28735 raft_consensus.cc:2243] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.189422 28735 raft_consensus.cc:2272] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.190191 28735 ts_tablet_manager.cc:1916] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.193573 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e31e11048b494ebe1fd4d89488b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.199275 28917 raft_consensus.cc:2804] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.200292 28736 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.200585 28736 ts_tablet_manager.cc:1403] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.005s
I20260320 15:27:25.200913 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_462a9e5f58364d459620a9eec9c2b570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.201725 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_462a9e5f58364d459620a9eec9c2b570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.201870 28736 raft_consensus.cc:359] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.202175 28736 raft_consensus.cc:385] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.202315 28736 raft_consensus.cc:740] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.202684 28736 consensus_queue.cc:260] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.202929 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 533105ad6e054fe3a4db32855bc688bd (DEFAULT_TABLE table=default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.205011 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd with delete_type TABLET_DATA_DELETED (replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.206789 28729 raft_consensus.cc:2243] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.206995 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f05205970cd249b68d138b904ba61c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.207615 28426 tablet_service.cc:1511] Processing CreateTablet for tablet c3bc4dd47817490d88725127b456cc70 (DEFAULT_TABLE table=default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.207783 28729 pending_rounds.cc:70] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:25.208153 28729 pending_rounds.cc:77] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7266387741447237632 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "75775842ffbb4ff1a61ba25bf6906c4b" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_a30a799b46f848b98910771717144df3" dest_uuid: "3ce5383d28b347ab979765b880e5773c" new_extra_config { } }
I20260320 15:27:25.209226 28729 raft_consensus.cc:2272] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.209424 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.209775 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5beaa1bc1e4465be34c8e86f131559 (DEFAULT_TABLE table=default.Table_a341aa1a6fbe4dedb77cedb3339ea59d [id=ba68ceb7a1404bcf90d28969990083dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.210166 28736 ts_tablet_manager.cc:1434] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260320 15:27:25.217541 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 80810790e42947a69ce6140feb9f7cbc (DEFAULT_TABLE table=default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 [id=a6da90b5243e4ddc93342b80dac5e784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.218055 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80810790e42947a69ce6140feb9f7cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.221405 28588 raft_consensus.cc:2468] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:25.226189 28048 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2ca38bc990634e21a2c38dd840488f8c (table default.Table_6892575e01594ece9a265373580a05bc [id=96f573af716c4ff5bd62ec012e593f9e]). Expected version 2 got 0
I20260320 15:27:25.228027 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5beaa1bc1e4465be34c8e86f131559 (DEFAULT_TABLE table=default.Table_a341aa1a6fbe4dedb77cedb3339ea59d [id=ba68ceb7a1404bcf90d28969990083dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.228592 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5beaa1bc1e4465be34c8e86f131559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.227394 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_8ad5a03705764f249c6964341f957801"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.229143 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ad5a03705764f249c6964341f957801 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.230672 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet d57566e7907c4783aafce347e46b89e3 with delete_type TABLET_DATA_DELETED (replaced by tablet 4c6f8c20b5904eafb9a62d14bb155c82) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.229323 28587 raft_consensus.cc:2468] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:25.233381 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 (DEFAULT_TABLE table=default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.233987 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd3032950254e6f9db8da35dbb1acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.235450 29037 raft_consensus.cc:2804] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.240276 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 533105ad6e054fe3a4db32855bc688bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.240440 28736 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.241140 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 (DEFAULT_TABLE table=default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.241223 28382 leader_election.cc:304] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:25.241564 29025 raft_consensus.cc:2804] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.241165 28917 raft_consensus.cc:697] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:25.241725 29025 raft_consensus.cc:697] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:25.242107 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 (DEFAULT_TABLE table=default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.241963 29025 consensus_queue.cc:237] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.241962 28917 consensus_queue.cc:237] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.246801 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5beaa1bc1e4465be34c8e86f131559 (DEFAULT_TABLE table=default.Table_a341aa1a6fbe4dedb77cedb3339ea59d [id=ba68ceb7a1404bcf90d28969990083dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.247484 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5beaa1bc1e4465be34c8e86f131559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.250690 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bd527e7856bf48069d98d45db0adac7e" } modify_external_catalogs: true
I20260320 15:27:25.250923 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bd527e7856bf48069d98d45db0adac7e" } modify_external_catalogs: true
W20260320 15:27:25.251394 28524 leader_election.cc:343] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:25.252751 28734 ts_tablet_manager.cc:1434] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.370s user 0.024s sys 0.016s
I20260320 15:27:25.253356 28734 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:25.255688 28524 leader_election.cc:343] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:25.263047 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3bc4dd47817490d88725127b456cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.266798 28736 tablet_bootstrap.cc:654] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.267673 28050 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2ca38bc990634e21a2c38dd840488f8c (table default.Table_6892575e01594ece9a265373580a05bc [id=96f573af716c4ff5bd62ec012e593f9e]). Expected version 2 got 0
I20260320 15:27:25.268599 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa5f6fe801b4452bf294a9ae23bb163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.268623 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_83f833bca071404790ce7888026891b7" } new_table_name: "default.Table_50cb6ab7e16b477dbb7fec9d108e6d48" modify_external_catalogs: true
W20260320 15:27:25.268332 28584 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.015s sys 0.023s
I20260320 15:27:25.270447 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 80810790e42947a69ce6140feb9f7cbc (DEFAULT_TABLE table=default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 [id=a6da90b5243e4ddc93342b80dac5e784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.271692 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd3032950254e6f9db8da35dbb1acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.271937 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.276307 28729 ts_tablet_manager.cc:1916] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.267794 29117 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:25.275904 28280 tablet_service.cc:1511] Processing CreateTablet for tablet f8f70142c3a449439a0a475bddafdd40 (DEFAULT_TABLE table=default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b [id=01294509a81747008737b6c3ffae50b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.277252 28284 tablet_service.cc:1511] Processing CreateTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede (DEFAULT_TABLE table=default.Table_ba23f221189b481191026b461623223b [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.283854 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:25.286659 29035 raft_consensus.cc:493] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.286954 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f3ee3c611d41b483bd35f868042d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.286900 29035 raft_consensus.cc:515] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.287844 29035 leader_election.cc:290] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.287809 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_326fcace90ef4e0ba553655d7b3930ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.288581 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_326fcace90ef4e0ba553655d7b3930ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.289569 28736 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.290398 28730 ts_tablet_manager.cc:1929] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:25.290566 28730 log.cc:1199] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6b6165eab6b0442fa9d94ce577c1b43c
I20260320 15:27:25.291249 28730 ts_tablet_manager.cc:1950] T 6b6165eab6b0442fa9d94ce577c1b43c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:25.291889 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 234) took 1444 ms. Trace:
I20260320 15:27:25.292048 28730 rpcz_store.cc:276] 0320 15:27:23.847692 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.847760 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:24.741692 (+893932us) ts_tablet_manager.cc:1151] Deleting tablet 6b6165eab6b0442fa9d94ce577c1b43c
0320 15:27:24.741703 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.025234 (+283531us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.290225 (+264991us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.291843 (+ 1618us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":881107}
I20260320 15:27:25.292543 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 235) took 1115 ms. Trace:
I20260320 15:27:25.292130 28736 ts_tablet_manager.cc:1403] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.051s user 0.004s sys 0.001s
I20260320 15:27:25.292658 28730 rpcz_store.cc:276] 0320 15:27:24.177202 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.177278 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:25.292374 (+1115096us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:25.292398 (+ 24us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.292507 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1097677}
I20260320 15:27:25.293000 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 236) took 1043 ms. Trace:
I20260320 15:27:25.293118 28730 rpcz_store.cc:276] 0320 15:27:24.249839 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.250026 (+ 187us) service_pool.cc:225] Handling call
0320 15:27:25.292878 (+1042852us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:25.292886 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.292969 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1017390}
I20260320 15:27:25.293409 28730 tablet_replica.cc:333] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:25.294592 28584 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.551s user 0.015s sys 0.023s
I20260320 15:27:25.302377 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f70142c3a449439a0a475bddafdd40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.302858 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5618796f1a4ff3ab4c1209dad0aede. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:25.306299 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d78ab2f7e0d7413ea9049dba9cbd8633 has no consensus state
I20260320 15:27:25.310196 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4fa9df7189e84297bb2b7b04ae79052f" } new_table_name: "default.Table_bc3aedc79a434d7b95351602c46df82a" modify_external_catalogs: true
I20260320 15:27:25.331588 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f38d66983ac842a4bb44792143adeeca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.332428 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f38d66983ac842a4bb44792143adeeca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.342667 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.342625 28736 raft_consensus.cc:359] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.343386 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet f591c45d9d73442ab1f0144ddfc8c48b
I20260320 15:27:25.344089 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet 81a1e0957439458083e3a86ac2f01b1e from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:25.349372 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_30e46b102a5d409895faee96f35b2b36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.350019 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30e46b102a5d409895faee96f35b2b36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.345005 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.361210 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet d547c032332d4d109222b4743fa28561
I20260320 15:27:25.361882 28045 master_service.cc:954] ReplaceTablet: received request to replace tablet 8a18f32c29224bd883ceb20658dff228 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:25.362705 28736 raft_consensus.cc:385] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.362872 28736 raft_consensus.cc:740] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.363184 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet d547c032332d4d109222b4743fa28561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.349001 28735 ts_tablet_manager.cc:1929] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:25.366889 29134 raft_consensus.cc:493] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.367143 29134 raft_consensus.cc:515] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.368865 29134 leader_election.cc:290] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.369187 29134 raft_consensus.cc:493] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.369417 29134 raft_consensus.cc:515] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.370486 29134 leader_election.cc:290] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.373087 29035 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.373804 28047 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 82e93cd92bfe40f2989408c66f04ad0e
I20260320 15:27:25.374004 29035 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.374735 29035 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.375579 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c1dcc311a14b33a289ae2b8a485d2b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:25.375789 28303 raft_consensus.cc:2468] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:25.376426 28524 leader_election.cc:304] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.377138 28735 log.cc:1199] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/da957aed322f46acaf5876c567c2ff94
W20260320 15:27:25.377264 28524 leader_election.cc:343] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: f640a6ae307140d2aee3992333e96d84
I20260320 15:27:25.377593 29035 raft_consensus.cc:493] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.377754 29035 raft_consensus.cc:515] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.378407 29035 leader_election.cc:290] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.378732 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_dd29804c0ff542cf8f2d36b963880b98" } new_table_name: "default.Table_a32711390c664fe387579cb227f11e85" modify_external_catalogs: true
I20260320 15:27:25.378962 28736 consensus_queue.cc:260] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.380445 28584 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.383999 28736 ts_tablet_manager.cc:1434] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.091s user 0.002s sys 0.001s
I20260320 15:27:25.383792 29134 raft_consensus.cc:2804] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:25.385465 29134 raft_consensus.cc:493] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:25.385593 29134 raft_consensus.cc:3060] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.389487 28733 tablet_bootstrap.cc:492] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:25.389659 28733 ts_tablet_manager.cc:1403] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.598s user 0.010s sys 0.003s
I20260320 15:27:25.390616 28733 raft_consensus.cc:359] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.390820 28733 raft_consensus.cc:385] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.390914 28733 raft_consensus.cc:740] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.391119 28733 consensus_queue.cc:260] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.384644 28735 ts_tablet_manager.cc:1950] T da957aed322f46acaf5876c567c2ff94 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:25.391672 28733 ts_tablet_manager.cc:1434] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:25.391991 28733 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:25.393735 28733 tablet_bootstrap.cc:654] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.395841 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet da957aed322f46acaf5876c567c2ff94 (table default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]) successfully deleted
W20260320 15:27:25.401283 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea6ff59d2ffe4d579e6db2375eb11414 already in progress: creating tablet
I20260320 15:27:25.401851 28735 tablet_replica.cc:333] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:25.402246 28735 raft_consensus.cc:2243] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.402570 28735 raft_consensus.cc:2272] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.403307 28735 ts_tablet_manager.cc:1916] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.406260 28054 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 3 got 2
I20260320 15:27:25.407461 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_248df89af9b1443e8ef145d0cf33ae62" } new_table_name: "default.Table_4495058468dd4441be0cbc0668f6bb8b" modify_external_catalogs: true
I20260320 15:27:25.405153 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.416563 28730 raft_consensus.cc:2243] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.417344 28730 raft_consensus.cc:2272] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.418851 28730 ts_tablet_manager.cc:1916] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.422587 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f3ee3c611d41b483bd35f868042d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.423223 28554 tablet_service.cc:1511] Processing CreateTablet for tablet f8f70142c3a449439a0a475bddafdd40 (DEFAULT_TABLE table=default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b [id=01294509a81747008737b6c3ffae50b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.423985 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f70142c3a449439a0a475bddafdd40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.425593 28736 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.426437 29037 raft_consensus.cc:697] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:25.426875 29037 consensus_queue.cc:237] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.430264 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f12e9706fc0f484da670771a01dd860c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.430936 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f12e9706fc0f484da670771a01dd860c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.431372 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 (DEFAULT_TABLE table=default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.431974 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd3032950254e6f9db8da35dbb1acc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.434484 29035 raft_consensus.cc:493] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.437050 29035 raft_consensus.cc:515] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.437885 29035 leader_election.cc:290] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.440222 28736 tablet_bootstrap.cc:654] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.442204 28051 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 3e0e4d1d9d1d4704b91a93686a2b40b5
I20260320 15:27:25.443753 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 80810790e42947a69ce6140feb9f7cbc (DEFAULT_TABLE table=default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 [id=a6da90b5243e4ddc93342b80dac5e784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.445106 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.276499 28411 tablet_service.cc:1511] Processing CreateTablet for tablet f8f70142c3a449439a0a475bddafdd40 (DEFAULT_TABLE table=default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b [id=01294509a81747008737b6c3ffae50b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.447186 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet d547c032332d4d109222b4743fa28561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.447598 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 6b6165eab6b0442fa9d94ce577c1b43c (table default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]) successfully deleted
W20260320 15:27:25.447885 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2165371f4b5448c7a2625926082ee4c6 already in progress: creating tablet
I20260320 15:27:25.451846 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_19d7fded966f4dfcad98437bdbc0e75c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.452625 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19d7fded966f4dfcad98437bdbc0e75c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:25.458540 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43a0fadc7dac4e21ad034e4cd2d70826 already in progress: creating tablet
I20260320 15:27:25.461292 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:25.461651 28587 raft_consensus.cc:2393] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
W20260320 15:27:25.462965 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e3f6d9a528946908891f77f951d23dd consensus state has no leader
I20260320 15:27:25.464346 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:25.465466 28239 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
W20260320 15:27:25.467299 28524 consensus_peers.cc:597] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81a1e0957439458083e3a86ac2f01b1e. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:25.467787 29128 raft_consensus.cc:493] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.468047 29128 raft_consensus.cc:515] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.469408 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:25.469652 28303 raft_consensus.cc:2468] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:25.469904 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5d4b16da449247bda955fe596e458307"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.470372 28524 leader_election.cc:304] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
W20260320 15:27:25.470595 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d4b16da449247bda955fe596e458307 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.472189 29105 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260320 15:27:25.473366 29094 raft_consensus.cc:2804] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:25.473646 29094 raft_consensus.cc:493] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:25.473821 29094 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:25.477424 29094 raft_consensus.cc:515] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.479398 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:25.479777 28300 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:25.481986 29128 leader_election.cc:290] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.483239 29134 raft_consensus.cc:515] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.485972 28300 raft_consensus.cc:2468] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 2.
I20260320 15:27:25.492456 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16e31e11048b494ebe1fd4d89488b0cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.495085 28046 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed]). Expected version 3 got 2
I20260320 15:27:25.495792 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f3ee3c611d41b483bd35f868042d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.496552 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da860b779a174adbb4fd6f7f7520348c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.496981 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a019d578836346b786ea9bfab466a42d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.497771 28524 leader_election.cc:304] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
W20260320 15:27:25.497917 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a019d578836346b786ea9bfab466a42d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.498530 29035 raft_consensus.cc:493] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.498783 29035 raft_consensus.cc:515] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.499516 29035 leader_election.cc:290] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.504061 29128 raft_consensus.cc:2804] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:25.504369 29128 raft_consensus.cc:697] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:25.504741 29128 consensus_queue.cc:237] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.508172 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.509035 29134 leader_election.cc:290] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:25.509109 28049 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ee4a0d559ce4dfea878bd3f1c03c09a consensus state has no leader
I20260320 15:27:25.512316 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f5cf94af44a94088b01b125cc76ec506" } modify_external_catalogs: true
I20260320 15:27:25.512581 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f5cf94af44a94088b01b125cc76ec506" } modify_external_catalogs: true
I20260320 15:27:25.515878 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.516338 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ae0ad80b6a44b91b18fa1e250f7dfbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:25.516727 28584 raft_consensus.cc:2468] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:25.518169 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet f591c45d9d73442ab1f0144ddfc8c48b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.509127 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c1dcc311a14b33a289ae2b8a485d2b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:25.514735 28239 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:25.520908 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 0c2339ff2c474a288876f32b85cecc72 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:25.521101 28239 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.521539 28300 raft_consensus.cc:3060] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.522205 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80810790e42947a69ce6140feb9f7cbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.522373 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80810790e42947a69ce6140feb9f7cbc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:25.524190 28240 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:25.524305 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ae0ad80b6a44b91b18fa1e250f7dfbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.524698 28444 raft_consensus.cc:2468] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:25.524374 28240 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.525035 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 442ca186c9d4449dade39271dec5c608 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:25.525261 28240 leader_election.cc:304] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.525596 29035 raft_consensus.cc:2749] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.525785 29035 raft_consensus.cc:3060] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.529279 29035 raft_consensus.cc:2749] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:25.532003 28524 leader_election.cc:343] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: f640a6ae307140d2aee3992333e96d84
I20260320 15:27:25.532265 28524 leader_election.cc:304] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:25.532824 28524 leader_election.cc:343] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 28c1dcc311a14b33a289ae2b8a485d2b
I20260320 15:27:25.533941 29037 raft_consensus.cc:493] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.534125 29037 raft_consensus.cc:515] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.534772 29037 leader_election.cc:290] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.534945 29037 raft_consensus.cc:2749] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.535622 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.537253 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:25.537509 28303 raft_consensus.cc:2468] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:25.538065 28524 leader_election.cc:304] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.538386 28300 raft_consensus.cc:2468] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:25.540088 28524 leader_election.cc:304] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.541213 29129 raft_consensus.cc:2804] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:25.541390 29129 raft_consensus.cc:493] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:25.541563 29129 raft_consensus.cc:3060] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.542073 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 8a18f32c29224bd883ceb20658dff228
I20260320 15:27:25.542616 28045 catalog_manager.cc:6741] ReplaceTablet: tablet 8a18f32c29224bd883ceb20658dff228 deleted and replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495
I20260320 15:27:25.543258 28415 tablet_service.cc:1511] Processing CreateTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede (DEFAULT_TABLE table=default.Table_ba23f221189b481191026b461623223b [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.547230 28736 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.538451 29084 consensus_queue.cc:1048] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.054s
I20260320 15:27:25.548552 29094 leader_election.cc:290] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.548889 29094 raft_consensus.cc:2804] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.549104 29094 raft_consensus.cc:697] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:25.552258 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet d547c032332d4d109222b4743fa28561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.553998 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.556540 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5beaa1bc1e4465be34c8e86f131559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.557803 28554 tablet_service.cc:1511] Processing CreateTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede (DEFAULT_TABLE table=default.Table_ba23f221189b481191026b461623223b [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.558049 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.560894 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_86f6401b78c540eea6cb427c9bd38f6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.561702 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86f6401b78c540eea6cb427c9bd38f6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.563318 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4389a8ff013e4930b2ebd4703a35f0da" } new_table_name: "default.Table_e26aaeaeae1047e8b3ea16c949628001" modify_external_catalogs: true
I20260320 15:27:25.564988 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f70142c3a449439a0a475bddafdd40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.573135 28736 ts_tablet_manager.cc:1403] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.132s user 0.005s sys 0.000s
W20260320 15:27:25.574200 28524 consensus_peers.cc:597] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81a1e0957439458083e3a86ac2f01b1e. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:25.574795 28524 leader_election.cc:343] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 28c1dcc311a14b33a289ae2b8a485d2b
I20260320 15:27:25.575202 28736 raft_consensus.cc:359] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.575484 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.575845 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5618796f1a4ff3ab4c1209dad0aede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.578569 28444 raft_consensus.cc:2468] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:25.579212 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "203c619513a146f49a4825962f169f45" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:25.582365 29129 raft_consensus.cc:515] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.582729 28927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33364 (request call id 0) took 2280 ms. Trace:
I20260320 15:27:25.583137 29129 leader_election.cc:290] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.583369 29106 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 07e8bb8819eb4cb4892e0cba1f758892)
I20260320 15:27:25.583684 29106 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:25.583714 29048 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:25.585530 29048 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:25.586838 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e8bb8819eb4cb4892e0cba1f758892. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:25.587249 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:25.587013 29127 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:25.587384 29127 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:25.588766 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.589742 29106 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:25.549441 29094 consensus_queue.cc:237] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.604669 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5618796f1a4ff3ab4c1209dad0aede. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.606066 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.606377 28239 leader_election.cc:304] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.608994 28734 tablet_bootstrap.cc:654] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.610569 28733 tablet_bootstrap.cc:492] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:25.610869 28733 ts_tablet_manager.cc:1403] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.219s user 0.003s sys 0.001s
I20260320 15:27:25.614190 28733 raft_consensus.cc:359] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.615187 29035 raft_consensus.cc:2804] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:25.615312 29035 raft_consensus.cc:493] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:25.615422 29035 raft_consensus.cc:3060] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.618243 29035 raft_consensus.cc:515] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.615880 28584 raft_consensus.cc:1240] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 LEADER]: Rejecting Update request from peer d22e73471b9046d28d3cf63c905b11b7 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260320 15:27:25.618767 28736 raft_consensus.cc:385] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.618943 28736 raft_consensus.cc:740] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.619287 28736 consensus_queue.cc:260] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.620653 28917 consensus_queue.cc:1059] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:25.621013 28917 consensus_queue.cc:1243] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Peer 80636bb608a1431b955576e296ff63bb log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260320 15:27:25.621275 28917 raft_consensus.cc:3055] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260320 15:27:25.621322 28730 ts_tablet_manager.cc:1929] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:25.621469 28730 log.cc:1199] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/75775842ffbb4ff1a61ba25bf6906c4b
I20260320 15:27:25.621513 28917 raft_consensus.cc:740] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:25.621956 28730 ts_tablet_manager.cc:1950] T 75775842ffbb4ff1a61ba25bf6906c4b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:25.622195 28733 raft_consensus.cc:385] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.623993 28735 ts_tablet_manager.cc:1929] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:25.624223 28735 log.cc:1199] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:25.624377 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:25.624625 28303 raft_consensus.cc:3060] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.624807 29122 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260320 15:27:25.626848 28733 raft_consensus.cc:740] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.626977 28735 ts_tablet_manager.cc:1950] T 6d1db9963c4645ae9aab72b4b22d6938 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:25.627223 28733 consensus_queue.cc:260] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.628850 28303 raft_consensus.cc:2468] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:25.629355 28524 leader_election.cc:304] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.632808 29035 leader_election.cc:290] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.633245 29134 raft_consensus.cc:2804] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.633507 29134 raft_consensus.cc:697] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:25.633924 29134 consensus_queue.cc:237] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.635550 28735 tablet_replica.cc:333] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:25.635828 28735 raft_consensus.cc:2243] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.636104 28735 raft_consensus.cc:2272] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.636763 28735 ts_tablet_manager.cc:1916] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:25.641829 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 116561us
W20260320 15:27:25.642225 28240 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:25.642489 28240 leader_election.cc:304] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:25.643260 28240 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.636786 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ae0ad80b6a44b91b18fa1e250f7dfbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:25.643461 28240 leader_election.cc:304] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.643733 29105 tablet.cc:1718] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Skipping requested alter to schema version 2, tablet already version 2
I20260320 15:27:25.643764 28584 raft_consensus.cc:3060] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.644806 29035 raft_consensus.cc:2749] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.645026 29035 raft_consensus.cc:2749] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.646250 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6d1db9963c4645ae9aab72b4b22d6938 (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]) successfully deleted
I20260320 15:27:25.646804 29146 raft_consensus.cc:493] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.647042 29146 raft_consensus.cc:515] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.643590 29122 tablet.cc:1718] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Skipping requested alter to schema version 2, tablet already version 2
I20260320 15:27:25.648269 29146 leader_election.cc:290] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.641795 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d57566e7907c4783aafce347e46b89e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:25.649535 29117 tablet.cc:1723] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260320 15:27:25.649577 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:25.650209 28239 leader_election.cc:343] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.654587 28584 raft_consensus.cc:2468] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:25.655265 28239 leader_election.cc:304] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:25.658301 28733 ts_tablet_manager.cc:1434] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.047s user 0.001s sys 0.001s
I20260320 15:27:25.658826 28733 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:25.658883 28447 raft_consensus.cc:3060] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.659783 29146 raft_consensus.cc:2804] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:25.665046 29149 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260320 15:27:25.670186 29146 raft_consensus.cc:697] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:25.670454 28736 ts_tablet_manager.cc:1434] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.097s user 0.002s sys 0.003s
I20260320 15:27:25.676040 28736 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.670083 28917 consensus_queue.cc:260] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.683046 28736 tablet_bootstrap.cc:654] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:25.695145 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 20a7164db1e0447fb424d8e46b747f52 consensus state has no leader
I20260320 15:27:25.695997 29144 tablet.cc:1718] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Skipping requested alter to schema version 2, tablet already version 2
I20260320 15:27:25.696780 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ae0ad80b6a44b91b18fa1e250f7dfbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:25.696893 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.697027 28444 raft_consensus.cc:3060] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:25.697567 28904 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "80636bb608a1431b955576e296ff63bb" instance_seqno: 1774020437674462) as {username='slave'} at 127.27.74.3:43105; Asking this server to re-register.
I20260320 15:27:25.700587 28444 raft_consensus.cc:2468] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:25.702517 29144 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:39582 (request call id 275) took 1044 ms. Trace:
I20260320 15:27:25.702823 29144 rpcz_store.cc:276] 0320 15:27:24.657854 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.657994 (+ 140us) service_pool.cc:225] Handling call
0320 15:27:25.702462 (+1044468us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:24.748578 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0320 15:27:24.748627 (+ 49us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0320 15:27:25.633255 (+884628us) alter_schema_op.cc:66] Acquired schema lock
0320 15:27:25.633268 (+ 13us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0320 15:27:25.633326 (+ 58us) alter_schema_op.cc:128] START. Timestamp: 1774020445633261
0320 15:27:25.633330 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:25.633544 (+ 214us) log.cc:844] Serialized 173 byte log entry
0320 15:27:25.642653 (+ 9109us) op_driver.cc:464] REPLICATION: finished
0320 15:27:25.695887 (+ 53234us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0320 15:27:25.702121 (+ 6234us) log.cc:844] Serialized 94 byte log entry
0320 15:27:25.702418 (+ 297us) alter_schema_op.cc:73] Released schema lock
0320 15:27:25.702424 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":53126,"prepare.queue_time_us":58353,"prepare.run_cpu_time_us":35990,"prepare.run_wall_time_us":885079,"raft.queue_time_us":2554,"raft.run_cpu_time_us":550,"raft.run_wall_time_us":1920,"replication_time_us":9281,"threads_started":1}]]}
I20260320 15:27:25.670616 29146 consensus_queue.cc:237] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.689399 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.691565 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:25.714267 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 7c4b53fb135741efad5d3e59a6d4eb3e from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:25.704233 28240 leader_election.cc:343] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.714757 28240 leader_election.cc:304] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.716030 29146 raft_consensus.cc:2749] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.728878 28735 ts_tablet_manager.cc:1929] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:25.729349 28917 raft_consensus.cc:3060] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:25.742204 28735 log.cc:1199] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/203c619513a146f49a4825962f169f45
I20260320 15:27:25.742799 28447 raft_consensus.cc:2468] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:25.737427 29105 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:25.730067 28736 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.743970 28736 ts_tablet_manager.cc:1403] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.001s
I20260320 15:27:25.748553 28736 raft_consensus.cc:359] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.749138 28736 raft_consensus.cc:385] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.749336 28736 raft_consensus.cc:740] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.749715 28736 consensus_queue.cc:260] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.750635 28736 ts_tablet_manager.cc:1434] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:25.751165 28736 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.756371 28634 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:25.757616 28736 tablet_bootstrap.cc:654] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.758422 29157 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:25.760699 29146 raft_consensus.cc:493] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.760946 29146 raft_consensus.cc:515] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.761683 29146 leader_election.cc:290] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:25.765486 28240 leader_election.cc:343] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 6b6165eab6b0442fa9d94ce577c1b43c
I20260320 15:27:25.769161 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 238) took 1514 ms. Trace:
I20260320 15:27:25.769482 28730 rpcz_store.cc:276] 0320 15:27:24.254963 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.255073 (+ 110us) service_pool.cc:225] Handling call
0320 15:27:25.293342 (+1038269us) ts_tablet_manager.cc:1151] Deleting tablet 75775842ffbb4ff1a61ba25bf6906c4b
0320 15:27:25.293350 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.417048 (+123698us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x6160000a3054. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965ce2f7e0 00007f965e180dc3 00007f9660d44a77 00007f9660d77b44 00007f9660d58560 00007f9660d582ed 0000000000a8b58c 00007f9650f75f5f 00007f9650f7927b 00007f9650f790f8
0320 15:27:25.537737 (+120689us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.621168 (+ 83431us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.749349 (+128181us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x6160000a3054. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965ce2f66a 00007f965cdeda14 00007f965e182c16 00007f9660d44e74 00007f9660d77b44 00007f9660d58560 00007f9660d582ed 0000000000a8b58c 00007f9650f75f5f 00007f9650f7927b
0320 15:27:25.769090 (+ 19741us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":422123648,"tablet-delete.queue_time_us":1017732}
I20260320 15:27:25.751775 28735 ts_tablet_manager.cc:1950] T 203c619513a146f49a4825962f169f45 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:25.770068 28239 leader_election.cc:343] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 6b6165eab6b0442fa9d94ce577c1b43c
I20260320 15:27:25.773387 28239 leader_election.cc:304] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.773766 29146 raft_consensus.cc:2749] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.780340 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 271) took 1144 ms. Trace:
I20260320 15:27:25.780534 28735 rpcz_store.cc:276] 0320 15:27:24.635937 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.651041 (+ 15104us) service_pool.cc:225] Handling call
0320 15:27:25.635442 (+984401us) ts_tablet_manager.cc:1151] Deleting tablet 203c619513a146f49a4825962f169f45
0320 15:27:25.635454 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.693489 (+ 58035us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.728681 (+ 35192us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.780142 (+ 51461us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":984059}
I20260320 15:27:25.781303 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 203c619513a146f49a4825962f169f45 (table default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]) successfully deleted
I20260320 15:27:25.783847 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 272) took 1018 ms. Trace:
I20260320 15:27:25.785051 28729 ts_tablet_manager.cc:1929] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:25.784974 28735 rpcz_store.cc:276] 0320 15:27:24.765087 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.765147 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:25.783521 (+1018374us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:25.783586 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.783769 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1018057}
W20260320 15:27:25.785888 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet edb7b53bb7444a39b471218c6039542f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb7b53bb7444a39b471218c6039542f already in progress: creating tablet
I20260320 15:27:25.783149 29122 tablet.cc:1723] T 2548d7b8d31049519a85490772621352 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260320 15:27:25.787184 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c4b53fb135741efad5d3e59a6d4eb3e already in progress: creating tablet
I20260320 15:27:25.788949 29146 raft_consensus.cc:493] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.789208 29146 raft_consensus.cc:515] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.790088 29146 leader_election.cc:290] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:25.788801 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 26818e620ff649d18b671e4f4c7c7e74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26818e620ff649d18b671e4f4c7c7e74 already in progress: creating tablet
I20260320 15:27:25.791775 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:25.792402 28239 leader_election.cc:343] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.792870 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 244) took 1487 ms. Trace:
I20260320 15:27:25.792892 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]) successfully deleted
I20260320 15:27:25.792902 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.793151 28730 rpcz_store.cc:276] 0320 15:27:24.305088 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.305156 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:25.792694 (+1487538us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:25.792704 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.792813 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1487222}
W20260320 15:27:25.793632 28240 leader_election.cc:343] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.793792 28240 leader_election.cc:304] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.795037 29146 raft_consensus.cc:2749] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:25.795701 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7f0242a7968541f480367b45cda5d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f0242a7968541f480367b45cda5d9cc already in progress: creating tablet
I20260320 15:27:25.796032 28733 tablet_bootstrap.cc:654] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:25.796595 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 093ae84740d5468188b4a56fa16ded9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 093ae84740d5468188b4a56fa16ded9c already in progress: creating tablet
I20260320 15:27:25.796152 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 245) took 1430 ms. Trace:
I20260320 15:27:25.796958 28730 rpcz_store.cc:276] 0320 15:27:24.363360 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.363433 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:25.794077 (+1430644us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:25.794090 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.794217 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1430341}
W20260320 15:27:25.797338 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ed29146bff74e86a366d3c6329176f4 already in progress: creating tablet
I20260320 15:27:25.797668 28730 tablet_replica.cc:333] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:25.797778 28735 tablet_replica.cc:333] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
W20260320 15:27:25.798944 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet f70530cd63a0420fb51527759c951e31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f70530cd63a0420fb51527759c951e31 already in progress: creating tablet
I20260320 15:27:25.799580 28730 raft_consensus.cc:2243] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.799856 28730 raft_consensus.cc:2272] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.800438 29146 raft_consensus.cc:493] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.800541 28730 ts_tablet_manager.cc:1916] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.800670 29146 raft_consensus.cc:515] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.801424 29146 leader_election.cc:290] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.802172 28735 raft_consensus.cc:2243] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.802452 28735 raft_consensus.cc:2272] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.803129 28735 ts_tablet_manager.cc:1916] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.806109 28904 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" instance_seqno: 1774020437138719) as {username='slave'} at 127.27.74.2:50195; Asking this server to re-register.
I20260320 15:27:25.806254 28736 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.806417 28736 ts_tablet_manager.cc:1403] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.001s
I20260320 15:27:25.807425 28736 raft_consensus.cc:359] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.807768 28736 raft_consensus.cc:385] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.807901 28736 raft_consensus.cc:740] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.808274 28736 consensus_queue.cc:260] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:25.810880 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4c6f8c20b5904eafb9a62d14bb155c82 consensus state has no leader
I20260320 15:27:25.812204 28736 ts_tablet_manager.cc:1434] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:25.812342 28054 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343] alter complete (version 6)
I20260320 15:27:25.813030 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c75cd37c416143ef883cf261b71cff1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.813217 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:25.813858 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c75cd37c416143ef883cf261b71cff1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.814046 28904 ts_manager.cc:194] Registered new tserver with Master: 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.817390 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3c78548647f4450687bd63dcc8123436" } modify_external_catalogs: true
I20260320 15:27:25.817667 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3c78548647f4450687bd63dcc8123436" } modify_external_catalogs: true
W20260320 15:27:25.815801 28239 leader_election.cc:343] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.812958 28736 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.824397 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e25b7a014de24f8c8f22a3584fd0d367" } modify_external_catalogs: true
I20260320 15:27:25.824642 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e25b7a014de24f8c8f22a3584fd0d367" } modify_external_catalogs: true
I20260320 15:27:25.826042 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a33f108177064a298842573efef5829c" } modify_external_catalogs: true
I20260320 15:27:25.826253 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a33f108177064a298842573efef5829c" } modify_external_catalogs: true
I20260320 15:27:25.826819 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.827587 28493 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:25.827611 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d" } modify_external_catalogs: true
I20260320 15:27:25.827775 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d" } modify_external_catalogs: true
W20260320 15:27:25.828536 28240 leader_election.cc:343] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.828497 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_818e1205d4f34148aebe2a4206ca41a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.828687 28240 leader_election.cc:304] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:25.829073 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_818e1205d4f34148aebe2a4206ca41a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.829777 29146 raft_consensus.cc:2749] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:25.830883 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8b29771187ba40789c95c13f432c263c" } new_table_name: "default.Table_bc31ab0980a04953909f5c8d56ae39ee" modify_external_catalogs: true
I20260320 15:27:25.832403 28736 tablet_bootstrap.cc:654] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.832767 28729 log.cc:1199] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/75775842ffbb4ff1a61ba25bf6906c4b
I20260320 15:27:25.834179 28906 ts_manager.cc:194] Registered new tserver with Master: d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:25.836670 28733 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:25.836699 28133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 5 candidate_status { last_received { term: 4 index: 300 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:25.836957 28733 ts_tablet_manager.cc:1403] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.178s user 0.007s sys 0.000s
I20260320 15:27:25.838081 28733 raft_consensus.cc:359] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.838280 28733 raft_consensus.cc:385] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.838374 28733 raft_consensus.cc:740] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.838657 28733 consensus_queue.cc:260] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.839222 28733 ts_tablet_manager.cc:1434] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260320 15:27:25.839908 28038 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet d56de00174db4e2d8b1c7215038a04aa consensus state has no leader
I20260320 15:27:25.840314 28735 ts_tablet_manager.cc:1929] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:25.840658 28733 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:25.840984 28735 log.cc:1199] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6b6165eab6b0442fa9d94ce577c1b43c
I20260320 15:27:25.841696 28735 ts_tablet_manager.cc:1950] T 6b6165eab6b0442fa9d94ce577c1b43c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:25.842397 28063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 5 candidate_status { last_received { term: 4 index: 300 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:25.843448 28862 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee1caebf640d4ec0bf9b28c4f496ca18; no voters: 07e8bb8819eb4cb4892e0cba1f758892, 336af1a126ca4ab583c03527b890239a
I20260320 15:27:25.844655 29106 raft_consensus.cc:2749] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260320 15:27:25.847453 28729 ts_tablet_manager.cc:1950] T 75775842ffbb4ff1a61ba25bf6906c4b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:25.850069 28906 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3ce5383d28b347ab979765b880e5773c" instance_seqno: 1774020436584169) as {username='slave'} at 127.27.74.1:41749; Asking this server to re-register.
I20260320 15:27:25.853489 28350 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:25.853307 28047 catalog_manager.cc:5671] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:25.853889 28047 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:25.855934 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 252) took 1569 ms. Trace:
I20260320 15:27:25.856104 28729 rpcz_store.cc:276] 0320 15:27:24.278712 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.278776 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:24.829859 (+551083us) ts_tablet_manager.cc:1151] Deleting tablet 75775842ffbb4ff1a61ba25bf6906c4b
0320 15:27:24.829870 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.207672 (+377802us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x6160000a1254. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff3892697e0 00007ff38a5badc3 00007ff38d17ea77 00007ff38d1b1b44 00007ff38d192560 00007ff38d1922ed 0000000000a8b58c 00007ff37d3aff5f 00007ff37d3b327b 00007ff37d3b30f8
0320 15:27:25.769794 (+562122us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.784866 (+ 15072us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.848326 (+ 63460us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":795883264,"tablet-delete.queue_time_us":550845}
I20260320 15:27:25.857064 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 75775842ffbb4ff1a61ba25bf6906c4b (table default.Table_a30a799b46f848b98910771717144df3 [id=51c653187cbb43cdbfccbaba8561a559]) successfully deleted
I20260320 15:27:25.858070 28734 tablet_bootstrap.cc:492] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:25.858335 28734 ts_tablet_manager.cc:1403] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.605s user 0.009s sys 0.009s
I20260320 15:27:25.858345 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 254) took 1573 ms. Trace:
I20260320 15:27:25.861675 29117 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:25.862488 28729 rpcz_store.cc:276] 0320 15:27:24.284797 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.284861 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:25.856628 (+1571767us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:25.856640 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.858126 (+ 1486us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1571476}
W20260320 15:27:25.863351 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43a0fadc7dac4e21ad034e4cd2d70826 already in progress: creating tablet
I20260320 15:27:25.863790 28734 raft_consensus.cc:359] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.865298 29105 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:25.865767 28906 ts_manager.cc:194] Registered new tserver with Master: 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:25.868754 28733 tablet_bootstrap.cc:654] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.868952 28734 raft_consensus.cc:385] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.869266 28734 raft_consensus.cc:740] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.871680 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 260) took 1421 ms. Trace:
I20260320 15:27:25.871680 28734 consensus_queue.cc:260] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.871833 28729 rpcz_store.cc:276] 0320 15:27:24.447781 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.448358 (+ 577us) service_pool.cc:225] Handling call
0320 15:27:25.862823 (+1414465us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:25.862834 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.869559 (+ 6725us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1412522}
W20260320 15:27:25.872565 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f70530cd63a0420fb51527759c951e31 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f70530cd63a0420fb51527759c951e31 already in progress: creating tablet
I20260320 15:27:25.872928 28419 tablet_service.cc:1511] Processing CreateTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f (DEFAULT_TABLE table=default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.874562 28729 tablet_replica.cc:333] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:25.875377 28734 ts_tablet_manager.cc:1434] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260320 15:27:25.876046 28734 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:25.880743 28730 ts_tablet_manager.cc:1929] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:25.881381 28730 log.cc:1199] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/da957aed322f46acaf5876c567c2ff94
I20260320 15:27:25.884157 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd8f5ccbd35418684fd27459c7a6b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.885136 28280 tablet_service.cc:1511] Processing CreateTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f (DEFAULT_TABLE table=default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.885578 28730 ts_tablet_manager.cc:1950] T da957aed322f46acaf5876c567c2ff94 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:25.885668 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd8f5ccbd35418684fd27459c7a6b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.885994 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 247) took 1500 ms. Trace:
I20260320 15:27:25.886121 28730 rpcz_store.cc:276] 0320 15:27:24.385251 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.385777 (+ 526us) service_pool.cc:225] Handling call
0320 15:27:25.797568 (+1411791us) ts_tablet_manager.cc:1151] Deleting tablet da957aed322f46acaf5876c567c2ff94
0320 15:27:25.797583 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.831417 (+ 33834us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.861149 (+ 29732us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.885953 (+ 24804us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1411438}
I20260320 15:27:25.886492 28730 tablet_replica.cc:333] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:25.886704 28730 raft_consensus.cc:2243] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:25.886979 28730 pending_rounds.cc:70] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:25.887058 28730 pending_rounds.cc:77] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387742851166208 op_type: NO_OP noop_request { }
I20260320 15:27:25.887172 28730 raft_consensus.cc:2889] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:25.887307 28730 raft_consensus.cc:2272] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.887915 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 23eda54577404cc6900cb3ae348604d4 (DEFAULT_TABLE table=default.Table_36123e5c9fed4e51bfab33f47ed1d82b [id=bf25edfb9f0e4924aadf3e85e57fab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.888370 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23eda54577404cc6900cb3ae348604d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.892138 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 60d0b36f462945cbbca7e389edf494d3 (DEFAULT_TABLE table=default.Table_c2a40da69e9d4735b3dd5e71d419b00b [id=cfa06a2552dd47ca947741764a0bf762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.892638 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d0b36f462945cbbca7e389edf494d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.582893 28927 rpcz_store.cc:276] 0320 15:27:23.302143 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:23.302239 (+ 96us) service_pool.cc:225] Handling call
0320 15:27:23.868144 (+565905us) raft_consensus.cc:1471] Updating replica for 274 ops
0320 15:27:24.097354 (+229210us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:24.097360 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:24.098018 (+ 658us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:24.098022 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:24.114332 (+ 16310us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:24.114336 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:24.209223 (+ 94887us) raft_consensus.cc:1534] Early marking committed up to index 0
0320 15:27:24.209232 (+ 9us) raft_consensus.cc:1539] Triggering prepare for 274 ops
0320 15:27:24.209713 (+ 481us) write_op.cc:270] Start()
0320 15:27:24.209781 (+ 68us) write_op.cc:276] Timestamp: P: 1774020438600220 usec, L: 0
0320 15:27:24.211004 (+ 1223us) write_op.cc:270] Start()
0320 15:27:24.211039 (+ 35us) write_op.cc:276] Timestamp: P: 1774020438601020 usec, L: 0
0320 15:27:24.211732 (+ 693us) write_op.cc:270] Start()
0320 15:27:24.211762 (+ 30us) write_op.cc:276] Timestamp: P: 1774020438602465 usec, L: 0
0320 15:27:24.212421 (+ 659us) write_op.cc:270] Start()
0320 15:27:24.212445 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438603086 usec, L: 0
0320 15:27:24.213056 (+ 611us) write_op.cc:270] Start()
0320 15:27:24.213080 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438604365 usec, L: 0
0320 15:27:24.213618 (+ 538us) write_op.cc:270] Start()
0320 15:27:24.213642 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438604894 usec, L: 0
0320 15:27:24.214215 (+ 573us) write_op.cc:270] Start()
0320 15:27:24.214272 (+ 57us) write_op.cc:276] Timestamp: P: 1774020438605484 usec, L: 0
0320 15:27:24.214875 (+ 603us) write_op.cc:270] Start()
0320 15:27:24.214901 (+ 26us) write_op.cc:276] Timestamp: P: 1774020438606569 usec, L: 0
0320 15:27:24.271457 (+ 56556us) write_op.cc:270] Start()
0320 15:27:24.271494 (+ 37us) write_op.cc:276] Timestamp: P: 1774020438607438 usec, L: 0
0320 15:27:24.275191 (+ 3697us) write_op.cc:270] Start()
0320 15:27:24.275327 (+ 136us) write_op.cc:276] Timestamp: P: 1774020438608341 usec, L: 0
0320 15:27:24.302203 (+ 26876us) write_op.cc:270] Start()
0320 15:27:24.302256 (+ 53us) write_op.cc:276] Timestamp: P: 1774020438618297 usec, L: 0
0320 15:27:24.403683 (+101427us) write_op.cc:270] Start()
0320 15:27:24.403759 (+ 76us) write_op.cc:276] Timestamp: P: 1774020438623997 usec, L: 0
0320 15:27:24.406546 (+ 2787us) write_op.cc:270] Start()
0320 15:27:24.406604 (+ 58us) write_op.cc:276] Timestamp: P: 1774020438628144 usec, L: 0
0320 15:27:24.406860 (+ 256us) write_op.cc:270] Start()
0320 15:27:24.406887 (+ 27us) write_op.cc:276] Timestamp: P: 1774020438628857 usec, L: 0
0320 15:27:24.407050 (+ 163us) write_op.cc:270] Start()
0320 15:27:24.407076 (+ 26us) write_op.cc:276] Timestamp: P: 1774020438629486 usec, L: 0
0320 15:27:24.407267 (+ 191us) write_op.cc:270] Start()
0320 15:27:24.407295 (+ 28us) write_op.cc:276] Timestamp: P: 1774020438631064 usec, L: 0
0320 15:27:24.407494 (+ 199us) write_op.cc:270] Start()
0320 15:27:24.407578 (+ 84us) write_op.cc:276] Timestamp: P: 1774020438631694 usec, L: 0
0320 15:27:24.407761 (+ 183us) write_op.cc:270] Start()
0320 15:27:24.407786 (+ 25us) write_op.cc:276] Timestamp: P: 1774020438632483 usec, L: 0
0320 15:27:24.407952 (+ 166us) write_op.cc:270] Start()
0320 15:27:24.407974 (+ 22us) write_op.cc:276] Timestamp: P: 1774020438633346 usec, L: 0
0320 15:27:24.408131 (+ 157us) write_op.cc:270] Start()
0320 15:27:24.408156 (+ 25us) write_op.cc:276] Timestamp: P: 1774020438634091 usec, L: 0
0320 15:27:24.408306 (+ 150us) write_op.cc:270] Start()
0320 15:27:24.408369 (+ 63us) write_op.cc:276] Timestamp: P: 1774020438637574 usec, L: 0
0320 15:27:24.408552 (+ 183us) write_op.cc:270] Start()
0320 15:27:24.408578 (+ 26us) write_op.cc:276] Timestamp: P: 1774020438659983 usec, L: 0
0320 15:27:24.408732 (+ 154us) write_op.cc:270] Start()
0320 15:27:24.408755 (+ 23us) write_op.cc:276] Timestamp: P: 1774020438799922 usec, L: 0
0320 15:27:24.408916 (+ 161us) write_op.cc:270] Start()
0320 15:27:24.408940 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438803315 usec, L: 0
0320 15:27:24.409095 (+ 155us) write_op.cc:270] Start()
0320 15:27:24.409134 (+ 39us) write_op.cc:276] Timestamp: P: 1774020438805757 usec, L: 0
0320 15:27:24.409384 (+ 250us) write_op.cc:270] Start()
0320 15:27:24.409411 (+ 27us) write_op.cc:276] Timestamp: P: 1774020438823653 usec, L: 0
0320 15:27:24.409581 (+ 170us) write_op.cc:270] Start()
0320 15:27:24.409605 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438824375 usec, L: 0
0320 15:27:24.409785 (+ 180us) write_op.cc:270] Start()
0320 15:27:24.409810 (+ 25us) write_op.cc:276] Timestamp: P: 1774020438828478 usec, L: 0
0320 15:27:24.409972 (+ 162us) write_op.cc:270] Start()
0320 15:27:24.409997 (+ 25us) write_op.cc:276] Timestamp: P: 1774020438833567 usec, L: 0
0320 15:27:24.410171 (+ 174us) write_op.cc:270] Start()
0320 15:27:24.410195 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438834509 usec, L: 0
0320 15:27:24.410365 (+ 170us) write_op.cc:270] Start()
0320 15:27:24.410391 (+ 26us) write_op.cc:276] Timestamp: P: 1774020438861083 usec, L: 0
0320 15:27:24.410566 (+ 175us) write_op.cc:270] Start()
0320 15:27:24.410592 (+ 26us) write_op.cc:276] Timestamp: P: 1774020438862634 usec, L: 0
0320 15:27:24.410765 (+ 173us) write_op.cc:270] Start()
0320 15:27:24.410790 (+ 25us) write_op.cc:276] Timestamp: P: 1774020438863862 usec, L: 0
0320 15:27:24.410957 (+ 167us) write_op.cc:270] Start()
0320 15:27:24.410981 (+ 24us) write_op.cc:276] Timestamp: P: 1774020438865127 usec, L: 0
0320 15:27:24.411140 (+ 159us) write_op.cc:270] Start()
0320 15:27:24.411163 (+ 23us) write_op.cc:276] Timestamp: P: 1774020438885623 usec, L: 0
0320 15:27:24.411365 (+ 202us) write_op.cc:270] Start()
0320 15:27:24.411392 (+ 27us) write_op.cc:276] Timestamp: P: 1774020438896847 usec, L: 0
0320 15:27:24.659927 (+248535us) write_op.cc:270] Start()
0320 15:27:24.660810 (+ 883us) write_op.cc:276] Timestamp: P: 1774020438906129 usec, L: 0
0320 15:27:24.662089 (+ 1279us) write_op.cc:270] Start()
0320 15:27:24.662124 (+ 35us) write_op.cc:276] Timestamp: P: 1774020438909995 usec, L: 0
0320 15:27:24.662326 (+ 202us) write_op.cc:270] Start()
0320 15:27:24.662348 (+ 22us) write_op.cc:276] Timestamp: P: 1774020438922899 usec, L: 0
0320 15:27:24.662491 (+ 143us) write_op.cc:270] Start()
0320 15:27:24.662513 (+ 22us) write_op.cc:276] Timestamp: P: 1774020438985461 usec, L: 0
0320 15:27:24.662639 (+ 126us) write_op.cc:270] Start()
0320 15:27:24.662657 (+ 18us) write_op.cc:276] Timestamp: P: 1774020438987264 usec, L: 0
0320 15:27:24.662815 (+ 158us) write_op.cc:270] Start()
0320 15:27:24.662837 (+ 22us) write_op.cc:276] Timestamp: P: 1774020438991397 usec, L: 0
0320 15:27:24.662973 (+ 136us) write_op.cc:270] Start()
0320 15:27:24.662993 (+ 20us) write_op.cc:276] Timestamp: P: 1774020438992477 usec, L: 0
0320 15:27:24.670805 (+ 7812us) write_op.cc:270] Start()
0320 15:27:24.670848 (+ 43us) write_op.cc:276] Timestamp: P: 1774020438993604 usec, L: 0
0320 15:27:24.671103 (+ 255us) write_op.cc:270] Start()
0320 15:27:24.671134 (+ 31us) write_op.cc:276] Timestamp: P: 1774020438994176 usec, L: 0
0320 15:27:24.671339 (+ 205us) write_op.cc:270] Start()
0320 15:27:24.671359 (+ 20us) write_op.cc:276] Timestamp: P: 1774020438994684 usec, L: 0
0320 15:27:24.671521 (+ 162us) write_op.cc:270] Start()
0320 15:27:24.681164 (+ 9643us) write_op.cc:276] Timestamp: P: 1774020439003400 usec, L: 0
0320 15:27:24.681548 (+ 384us) write_op.cc:270] Start()
0320 15:27:24.681578 (+ 30us) write_op.cc:276] Timestamp: P: 1774020439069998 usec, L: 0
0320 15:27:24.681766 (+ 188us) write_op.cc:270] Start()
0320 15:27:24.681786 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439089018 usec, L: 0
0320 15:27:24.681903 (+ 117us) write_op.cc:270] Start()
0320 15:27:24.681922 (+ 19us) write_op.cc:276] Timestamp: P: 1774020439104860 usec, L: 0
0320 15:27:24.682070 (+ 148us) write_op.cc:270] Start()
0320 15:27:24.682088 (+ 18us) write_op.cc:276] Timestamp: P: 1774020439115461 usec, L: 0
0320 15:27:24.682274 (+ 186us) write_op.cc:270] Start()
0320 15:27:24.682300 (+ 26us) write_op.cc:276] Timestamp: P: 1774020439137441 usec, L: 0
0320 15:27:24.682445 (+ 145us) write_op.cc:270] Start()
0320 15:27:24.682464 (+ 19us) write_op.cc:276] Timestamp: P: 1774020439173301 usec, L: 0
0320 15:27:24.682596 (+ 132us) write_op.cc:270] Start()
0320 15:27:24.682616 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439191865 usec, L: 0
0320 15:27:24.682774 (+ 158us) write_op.cc:270] Start()
0320 15:27:24.682800 (+ 26us) write_op.cc:276] Timestamp: P: 1774020439209721 usec, L: 0
0320 15:27:24.682965 (+ 165us) write_op.cc:270] Start()
0320 15:27:24.682991 (+ 26us) write_op.cc:276] Timestamp: P: 1774020439230412 usec, L: 0
0320 15:27:24.683180 (+ 189us) write_op.cc:270] Start()
0320 15:27:24.683203 (+ 23us) write_op.cc:276] Timestamp: P: 1774020439244621 usec, L: 0
0320 15:27:24.700601 (+ 17398us) write_op.cc:270] Start()
0320 15:27:24.700644 (+ 43us) write_op.cc:276] Timestamp: P: 1774020439265301 usec, L: 0
0320 15:27:24.700834 (+ 190us) write_op.cc:270] Start()
0320 15:27:24.700860 (+ 26us) write_op.cc:276] Timestamp: P: 1774020439302221 usec, L: 0
0320 15:27:24.701011 (+ 151us) write_op.cc:270] Start()
0320 15:27:24.701032 (+ 21us) write_op.cc:276] Timestamp: P: 1774020439305547 usec, L: 0
0320 15:27:24.706060 (+ 5028us) write_op.cc:270] Start()
0320 15:27:24.706115 (+ 55us) write_op.cc:276] Timestamp: P: 1774020439310964 usec, L: 0
0320 15:27:24.708391 (+ 2276us) write_op.cc:270] Start()
0320 15:27:24.708425 (+ 34us) write_op.cc:276] Timestamp: P: 1774020439329719 usec, L: 0
0320 15:27:24.708607 (+ 182us) write_op.cc:270] Start()
0320 15:27:24.708636 (+ 29us) write_op.cc:276] Timestamp: P: 1774020439332178 usec, L: 0
0320 15:27:24.708772 (+ 136us) write_op.cc:270] Start()
0320 15:27:24.708791 (+ 19us) write_op.cc:276] Timestamp: P: 1774020439334253 usec, L: 0
0320 15:27:24.708945 (+ 154us) write_op.cc:270] Start()
0320 15:27:24.708966 (+ 21us) write_op.cc:276] Timestamp: P: 1774020439353158 usec, L: 0
0320 15:27:24.709127 (+ 161us) write_op.cc:270] Start()
0320 15:27:24.709164 (+ 37us) write_op.cc:276] Timestamp: P: 1774020439355153 usec, L: 0
0320 15:27:24.709331 (+ 167us) write_op.cc:270] Start()
0320 15:27:24.709352 (+ 21us) write_op.cc:276] Timestamp: P: 1774020439508201 usec, L: 0
0320 15:27:24.709500 (+ 148us) write_op.cc:270] Start()
0320 15:27:24.709522 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439508881 usec, L: 0
0320 15:27:24.709657 (+ 135us) write_op.cc:270] Start()
0320 15:27:24.709679 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439546598 usec, L: 0
0320 15:27:24.739268 (+ 29589us) write_op.cc:270] Start()
0320 15:27:24.739314 (+ 46us) write_op.cc:276] Timestamp: P: 1774020439556684 usec, L: 0
0320 15:27:24.739568 (+ 254us) write_op.cc:270] Start()
0320 15:27:24.739596 (+ 28us) write_op.cc:276] Timestamp: P: 1774020439569107 usec, L: 0
0320 15:27:24.739763 (+ 167us) write_op.cc:270] Start()
0320 15:27:24.739783 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439570210 usec, L: 0
0320 15:27:24.739928 (+ 145us) write_op.cc:270] Start()
0320 15:27:24.739949 (+ 21us) write_op.cc:276] Timestamp: P: 1774020439576111 usec, L: 0
0320 15:27:24.740104 (+ 155us) write_op.cc:270] Start()
0320 15:27:24.740126 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439608623 usec, L: 0
0320 15:27:24.740285 (+ 159us) write_op.cc:270] Start()
0320 15:27:24.740305 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439609791 usec, L: 0
0320 15:27:24.740437 (+ 132us) write_op.cc:270] Start()
0320 15:27:24.740458 (+ 21us) write_op.cc:276] Timestamp: P: 1774020439610894 usec, L: 0
0320 15:27:24.740614 (+ 156us) write_op.cc:270] Start()
0320 15:27:24.740634 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439612074 usec, L: 0
0320 15:27:24.759888 (+ 19254us) write_op.cc:270] Start()
0320 15:27:24.759924 (+ 36us) write_op.cc:276] Timestamp: P: 1774020439612973 usec, L: 0
0320 15:27:24.760124 (+ 200us) write_op.cc:270] Start()
0320 15:27:24.760191 (+ 67us) write_op.cc:276] Timestamp: P: 1774020439615673 usec, L: 0
0320 15:27:24.760405 (+ 214us) write_op.cc:270] Start()
0320 15:27:24.760448 (+ 43us) write_op.cc:276] Timestamp: P: 1774020439619677 usec, L: 0
0320 15:27:24.763447 (+ 2999us) write_op.cc:270] Start()
0320 15:27:24.763481 (+ 34us) write_op.cc:276] Timestamp: P: 1774020439622688 usec, L: 0
0320 15:27:24.763729 (+ 248us) write_op.cc:270] Start()
0320 15:27:24.763757 (+ 28us) write_op.cc:276] Timestamp: P: 1774020439681058 usec, L: 0
0320 15:27:24.766651 (+ 2894us) write_op.cc:270] Start()
0320 15:27:24.766687 (+ 36us) write_op.cc:276] Timestamp: P: 1774020439684334 usec, L: 0
0320 15:27:24.766923 (+ 236us) write_op.cc:270] Start()
0320 15:27:24.766954 (+ 31us) write_op.cc:276] Timestamp: P: 1774020439687278 usec, L: 0
0320 15:27:24.767140 (+ 186us) write_op.cc:270] Start()
0320 15:27:24.767179 (+ 39us) write_op.cc:276] Timestamp: P: 1774020439699266 usec, L: 0
0320 15:27:24.767360 (+ 181us) write_op.cc:270] Start()
0320 15:27:24.767394 (+ 34us) write_op.cc:276] Timestamp: P: 1774020439730118 usec, L: 0
0320 15:27:24.771492 (+ 4098us) write_op.cc:270] Start()
0320 15:27:24.771563 (+ 71us) write_op.cc:276] Timestamp: P: 1774020439737392 usec, L: 0
0320 15:27:24.771856 (+ 293us) write_op.cc:270] Start()
0320 15:27:24.771887 (+ 31us) write_op.cc:276] Timestamp: P: 1774020439773914 usec, L: 0
0320 15:27:24.772045 (+ 158us) write_op.cc:270] Start()
0320 15:27:24.772071 (+ 26us) write_op.cc:276] Timestamp: P: 1774020439775915 usec, L: 0
0320 15:27:24.772255 (+ 184us) write_op.cc:270] Start()
0320 15:27:24.772285 (+ 30us) write_op.cc:276] Timestamp: P: 1774020439794715 usec, L: 0
0320 15:27:24.772464 (+ 179us) write_op.cc:270] Start()
0320 15:27:24.772487 (+ 23us) write_op.cc:276] Timestamp: P: 1774020439799199 usec, L: 0
0320 15:27:24.772636 (+ 149us) write_op.cc:270] Start()
0320 15:27:24.772714 (+ 78us) write_op.cc:276] Timestamp: P: 1774020439807117 usec, L: 0
0320 15:27:24.772935 (+ 221us) write_op.cc:270] Start()
0320 15:27:24.772965 (+ 30us) write_op.cc:276] Timestamp: P: 1774020439807886 usec, L: 0
0320 15:27:24.773178 (+ 213us) write_op.cc:270] Start()
0320 15:27:24.773207 (+ 29us) write_op.cc:276] Timestamp: P: 1774020439808771 usec, L: 0
0320 15:27:24.773344 (+ 137us) write_op.cc:270] Start()
0320 15:27:24.773369 (+ 25us) write_op.cc:276] Timestamp: P: 1774020439859731 usec, L: 0
0320 15:27:24.773517 (+ 148us) write_op.cc:270] Start()
0320 15:27:24.773537 (+ 20us) write_op.cc:276] Timestamp: P: 1774020439860516 usec, L: 0
0320 15:27:24.773675 (+ 138us) write_op.cc:270] Start()
0320 15:27:24.773694 (+ 19us) write_op.cc:276] Timestamp: P: 1774020439868890 usec, L: 0
0320 15:27:24.773849 (+ 155us) write_op.cc:270] Start()
0320 15:27:24.773887 (+ 38us) write_op.cc:276] Timestamp: P: 1774020439869665 usec, L: 0
0320 15:27:24.774085 (+ 198us) write_op.cc:270] Start()
0320 15:27:24.774108 (+ 23us) write_op.cc:276] Timestamp: P: 1774020439872854 usec, L: 0
0320 15:27:24.774269 (+ 161us) write_op.cc:270] Start()
0320 15:27:24.774291 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439901264 usec, L: 0
0320 15:27:24.774459 (+ 168us) write_op.cc:270] Start()
0320 15:27:24.774482 (+ 23us) write_op.cc:276] Timestamp: P: 1774020439902667 usec, L: 0
0320 15:27:24.774654 (+ 172us) write_op.cc:270] Start()
0320 15:27:24.774676 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439903779 usec, L: 0
0320 15:27:24.774822 (+ 146us) write_op.cc:270] Start()
0320 15:27:24.774844 (+ 22us) write_op.cc:276] Timestamp: P: 1774020439943808 usec, L: 0
0320 15:27:24.775042 (+ 198us) write_op.cc:270] Start()
0320 15:27:24.775071 (+ 29us) write_op.cc:276] Timestamp: P: 1774020439946544 usec, L: 0
0320 15:27:24.775257 (+ 186us) write_op.cc:270] Start()
0320 15:27:24.775284 (+ 27us) write_op.cc:276] Timestamp: P: 1774020439947225 usec, L: 0
0320 15:27:24.795136 (+ 19852us) write_op.cc:270] Start()
0320 15:27:24.795199 (+ 63us) write_op.cc:276] Timestamp: P: 1774020439947820 usec, L: 0
0320 15:27:24.795488 (+ 289us) write_op.cc:270] Start()
0320 15:27:24.795548 (+ 60us) write_op.cc:276] Timestamp: P: 1774020439950939 usec, L: 0
0320 15:27:24.795729 (+ 181us) write_op.cc:270] Start()
0320 15:27:24.795752 (+ 23us) write_op.cc:276] Timestamp: P: 1774020439986083 usec, L: 0
0320 15:27:24.795909 (+ 157us) write_op.cc:270] Start()
0320 15:27:24.795933 (+ 24us) write_op.cc:276] Timestamp: P: 1774020439992812 usec, L: 0
0320 15:27:24.796120 (+ 187us) write_op.cc:270] Start()
0320 15:27:24.796145 (+ 25us) write_op.cc:276] Timestamp: P: 1774020440032268 usec, L: 0
0320 15:27:24.796312 (+ 167us) write_op.cc:270] Start()
0320 15:27:24.796334 (+ 22us) write_op.cc:276] Timestamp: P: 1774020440035415 usec, L: 0
0320 15:27:24.796527 (+ 193us) write_op.cc:270] Start()
0320 15:27:24.796549 (+ 22us) write_op.cc:276] Timestamp: P: 1774020440036836 usec, L: 0
0320 15:27:24.796700 (+ 151us) write_op.cc:270] Start()
0320 15:27:24.796721 (+ 21us) write_op.cc:276] Timestamp: P: 1774020440037975 usec, L: 0
0320 15:27:24.796892 (+ 171us) write_op.cc:270] Start()
0320 15:27:24.796913 (+ 21us) write_op.cc:276] Timestamp: P: 1774020440039568 usec, L: 0
0320 15:27:24.797067 (+ 154us) write_op.cc:270] Start()
0320 15:27:24.797089 (+ 22us) write_op.cc:276] Timestamp: P: 1774020440070004 usec, L: 0
0320 15:27:24.809736 (+ 12647us) write_op.cc:270] Start()
0320 15:27:24.809782 (+ 46us) write_op.cc:276] Timestamp: P: 1774020440089560 usec, L: 0
0320 15:27:24.810008 (+ 226us) write_op.cc:270] Start()
0320 15:27:24.810031 (+ 23us) write_op.cc:276] Timestamp: P: 1774020440102078 usec, L: 0
0320 15:27:24.827449 (+ 17418us) write_op.cc:270] Start()
0320 15:27:24.827671 (+ 222us) write_op.cc:276] Timestamp: P: 1774020440103616 usec, L: 0
0320 15:27:24.827975 (+ 304us) write_op.cc:270] Start()
0320 15:27:24.828008 (+ 33us) write_op.cc:276] Timestamp: P: 1774020440104646 usec, L: 0
0320 15:27:24.828199 (+ 191us) write_op.cc:270] Start()
0320 15:27:24.828220 (+ 21us) write_op.cc:276] Timestamp: P: 1774020440105923 usec, L: 0
0320 15:27:24.828376 (+ 156us) write_op.cc:270] Start()
0320 15:27:24.828394 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440108630 usec, L: 0
0320 15:27:24.828519 (+ 125us) write_op.cc:270] Start()
0320 15:27:24.828538 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440115991 usec, L: 0
0320 15:27:24.828654 (+ 116us) write_op.cc:270] Start()
0320 15:27:24.828671 (+ 17us) write_op.cc:276] Timestamp: P: 1774020440117199 usec, L: 0
0320 15:27:24.828793 (+ 122us) write_op.cc:270] Start()
0320 15:27:24.828809 (+ 16us) write_op.cc:276] Timestamp: P: 1774020440130346 usec, L: 0
0320 15:27:24.828952 (+ 143us) write_op.cc:270] Start()
0320 15:27:24.828970 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440130922 usec, L: 0
0320 15:27:24.829110 (+ 140us) write_op.cc:270] Start()
0320 15:27:24.829128 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440146242 usec, L: 0
0320 15:27:24.829260 (+ 132us) write_op.cc:270] Start()
0320 15:27:24.829277 (+ 17us) write_op.cc:276] Timestamp: P: 1774020440174677 usec, L: 0
0320 15:27:24.829481 (+ 204us) write_op.cc:270] Start()
0320 15:27:24.829503 (+ 22us) write_op.cc:276] Timestamp: P: 1774020440196719 usec, L: 0
0320 15:27:24.829695 (+ 192us) write_op.cc:270] Start()
0320 15:27:24.829716 (+ 21us) write_op.cc:276] Timestamp: P: 1774020440210136 usec, L: 0
0320 15:27:24.829880 (+ 164us) write_op.cc:270] Start()
0320 15:27:24.829905 (+ 25us) write_op.cc:276] Timestamp: P: 1774020440210844 usec, L: 0
0320 15:27:24.830047 (+ 142us) write_op.cc:270] Start()
0320 15:27:24.830067 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440212167 usec, L: 0
0320 15:27:24.830183 (+ 116us) write_op.cc:270] Start()
0320 15:27:24.830200 (+ 17us) write_op.cc:276] Timestamp: P: 1774020440230511 usec, L: 0
0320 15:27:24.830329 (+ 129us) write_op.cc:270] Start()
0320 15:27:24.830345 (+ 16us) write_op.cc:276] Timestamp: P: 1774020440261576 usec, L: 0
0320 15:27:24.830452 (+ 107us) write_op.cc:270] Start()
0320 15:27:24.830468 (+ 16us) write_op.cc:276] Timestamp: P: 1774020440392623 usec, L: 0
0320 15:27:24.830574 (+ 106us) write_op.cc:270] Start()
0320 15:27:24.830590 (+ 16us) write_op.cc:276] Timestamp: P: 1774020440399395 usec, L: 0
0320 15:27:24.830718 (+ 128us) write_op.cc:270] Start()
0320 15:27:24.830741 (+ 23us) write_op.cc:276] Timestamp: P: 1774020440449030 usec, L: 0
0320 15:27:24.830887 (+ 146us) write_op.cc:270] Start()
0320 15:27:24.830909 (+ 22us) write_op.cc:276] Timestamp: P: 1774020440587456 usec, L: 0
0320 15:27:24.831062 (+ 153us) write_op.cc:270] Start()
0320 15:27:24.831082 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440623818 usec, L: 0
0320 15:27:24.831214 (+ 132us) write_op.cc:270] Start()
0320 15:27:24.831233 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440648593 usec, L: 0
0320 15:27:24.831374 (+ 141us) write_op.cc:270] Start()
0320 15:27:24.831398 (+ 24us) write_op.cc:276] Timestamp: P: 1774020440661128 usec, L: 0
0320 15:27:24.831601 (+ 203us) write_op.cc:270] Start()
0320 15:27:24.831626 (+ 25us) write_op.cc:276] Timestamp: P: 1774020440708530 usec, L: 0
0320 15:27:24.831797 (+ 171us) write_op.cc:270] Start()
0320 15:27:24.831816 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440709390 usec, L: 0
0320 15:27:24.831947 (+ 131us) write_op.cc:270] Start()
0320 15:27:24.831966 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440710070 usec, L: 0
0320 15:27:24.832123 (+ 157us) write_op.cc:270] Start()
0320 15:27:24.832141 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440710652 usec, L: 0
0320 15:27:24.832268 (+ 127us) write_op.cc:270] Start()
0320 15:27:24.832286 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440711227 usec, L: 0
0320 15:27:24.832434 (+ 148us) write_op.cc:270] Start()
0320 15:27:24.832454 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440781821 usec, L: 0
0320 15:27:24.832607 (+ 153us) write_op.cc:270] Start()
0320 15:27:24.832627 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440782766 usec, L: 0
0320 15:27:24.832774 (+ 147us) write_op.cc:270] Start()
0320 15:27:24.832794 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440783387 usec, L: 0
0320 15:27:24.832949 (+ 155us) write_op.cc:270] Start()
0320 15:27:24.832968 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440862358 usec, L: 0
0320 15:27:24.833106 (+ 138us) write_op.cc:270] Start()
0320 15:27:24.833133 (+ 27us) write_op.cc:276] Timestamp: P: 1774020440863520 usec, L: 0
0320 15:27:24.833306 (+ 173us) write_op.cc:270] Start()
0320 15:27:24.833338 (+ 32us) write_op.cc:276] Timestamp: P: 1774020440864236 usec, L: 0
0320 15:27:24.833516 (+ 178us) write_op.cc:270] Start()
0320 15:27:24.833543 (+ 27us) write_op.cc:276] Timestamp: P: 1774020440865526 usec, L: 0
0320 15:27:24.833757 (+ 214us) write_op.cc:270] Start()
0320 15:27:24.833780 (+ 23us) write_op.cc:276] Timestamp: P: 1774020440866501 usec, L: 0
0320 15:27:24.833919 (+ 139us) write_op.cc:270] Start()
0320 15:27:24.833937 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440906141 usec, L: 0
0320 15:27:24.834089 (+ 152us) write_op.cc:270] Start()
0320 15:27:24.834112 (+ 23us) write_op.cc:276] Timestamp: P: 1774020440910574 usec, L: 0
0320 15:27:24.834256 (+ 144us) write_op.cc:270] Start()
0320 15:27:24.834275 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440941370 usec, L: 0
0320 15:27:24.834406 (+ 131us) write_op.cc:270] Start()
0320 15:27:24.834426 (+ 20us) write_op.cc:276] Timestamp: P: 1774020440942145 usec, L: 0
0320 15:27:24.834556 (+ 130us) write_op.cc:270] Start()
0320 15:27:24.834574 (+ 18us) write_op.cc:276] Timestamp: P: 1774020440942936 usec, L: 0
0320 15:27:24.834699 (+ 125us) write_op.cc:270] Start()
0320 15:27:24.834718 (+ 19us) write_op.cc:276] Timestamp: P: 1774020440977125 usec, L: 0
0320 15:27:24.834832 (+ 114us) write_op.cc:270] Start()
0320 15:27:24.834850 (+ 18us) write_op.cc:276] Timestamp: P: 1774020441023507 usec, L: 0
0320 15:27:24.834972 (+ 122us) write_op.cc:270] Start()
0320 15:27:24.834989 (+ 17us) write_op.cc:276] Timestamp: P: 1774020441196067 usec, L: 0
0320 15:27:24.835163 (+ 174us) write_op.cc:270] Start()
0320 15:27:24.835184 (+ 21us) write_op.cc:276] Timestamp: P: 1774020441209912 usec, L: 0
0320 15:27:24.835367 (+ 183us) write_op.cc:270] Start()
0320 15:27:24.835410 (+ 43us) write_op.cc:276] Timestamp: P: 1774020441214051 usec, L: 0
0320 15:27:24.909489 (+ 74079us) write_op.cc:270] Start()
0320 15:27:24.909536 (+ 47us) write_op.cc:276] Timestamp: P: 1774020441242148 usec, L: 0
0320 15:27:24.909782 (+ 246us) write_op.cc:270] Start()
0320 15:27:24.909816 (+ 34us) write_op.cc:276] Timestamp: P: 1774020441276440 usec, L: 0
0320 15:27:24.910030 (+ 214us) write_op.cc:270] Start()
0320 15:27:24.910057 (+ 27us) write_op.cc:276] Timestamp: P: 1774020441363504 usec, L: 0
0320 15:27:24.910256 (+ 199us) write_op.cc:270] Start()
0320 15:27:24.910344 (+ 88us) write_op.cc:276] Timestamp: P: 1774020441371564 usec, L: 0
0320 15:27:24.910557 (+ 213us) write_op.cc:270] Start()
0320 15:27:24.910580 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441387116 usec, L: 0
0320 15:27:24.910783 (+ 203us) write_op.cc:270] Start()
0320 15:27:24.910807 (+ 24us) write_op.cc:276] Timestamp: P: 1774020441389543 usec, L: 0
0320 15:27:24.910964 (+ 157us) write_op.cc:270] Start()
0320 15:27:24.910983 (+ 19us) write_op.cc:276] Timestamp: P: 1774020441393845 usec, L: 0
0320 15:27:24.911166 (+ 183us) write_op.cc:270] Start()
0320 15:27:24.911220 (+ 54us) write_op.cc:276] Timestamp: P: 1774020441394457 usec, L: 0
0320 15:27:24.911399 (+ 179us) write_op.cc:270] Start()
0320 15:27:24.911422 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441395050 usec, L: 0
0320 15:27:24.911633 (+ 211us) write_op.cc:270] Start()
0320 15:27:24.911656 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441395625 usec, L: 0
0320 15:27:24.911802 (+ 146us) write_op.cc:270] Start()
0320 15:27:24.911825 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441407887 usec, L: 0
0320 15:27:24.912014 (+ 189us) write_op.cc:270] Start()
0320 15:27:24.912043 (+ 29us) write_op.cc:276] Timestamp: P: 1774020441409068 usec, L: 0
0320 15:27:24.912232 (+ 189us) write_op.cc:270] Start()
0320 15:27:24.912261 (+ 29us) write_op.cc:276] Timestamp: P: 1774020441416433 usec, L: 0
0320 15:27:24.914183 (+ 1922us) write_op.cc:270] Start()
0320 15:27:24.914230 (+ 47us) write_op.cc:276] Timestamp: P: 1774020441417858 usec, L: 0
0320 15:27:24.914438 (+ 208us) write_op.cc:270] Start()
0320 15:27:24.914464 (+ 26us) write_op.cc:276] Timestamp: P: 1774020441419146 usec, L: 0
0320 15:27:24.914668 (+ 204us) write_op.cc:270] Start()
0320 15:27:24.914696 (+ 28us) write_op.cc:276] Timestamp: P: 1774020441420475 usec, L: 0
0320 15:27:24.914863 (+ 167us) write_op.cc:270] Start()
0320 15:27:24.914884 (+ 21us) write_op.cc:276] Timestamp: P: 1774020441421705 usec, L: 0
0320 15:27:24.915056 (+ 172us) write_op.cc:270] Start()
0320 15:27:24.915079 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441442845 usec, L: 0
0320 15:27:24.915254 (+ 175us) write_op.cc:270] Start()
0320 15:27:24.915277 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441465783 usec, L: 0
0320 15:27:24.915424 (+ 147us) write_op.cc:270] Start()
0320 15:27:24.915446 (+ 22us) write_op.cc:276] Timestamp: P: 1774020441467554 usec, L: 0
0320 15:27:24.915615 (+ 169us) write_op.cc:270] Start()
0320 15:27:24.915638 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441523859 usec, L: 0
0320 15:27:24.929488 (+ 13850us) write_op.cc:270] Start()
0320 15:27:24.929550 (+ 62us) write_op.cc:276] Timestamp: P: 1774020441534195 usec, L: 0
0320 15:27:24.929829 (+ 279us) write_op.cc:270] Start()
0320 15:27:24.929857 (+ 28us) write_op.cc:276] Timestamp: P: 1774020441536776 usec, L: 0
0320 15:27:24.930013 (+ 156us) write_op.cc:270] Start()
0320 15:27:24.930037 (+ 24us) write_op.cc:276] Timestamp: P: 1774020441540908 usec, L: 0
0320 15:27:24.930212 (+ 175us) write_op.cc:270] Start()
0320 15:27:24.930234 (+ 22us) write_op.cc:276] Timestamp: P: 1774020441550156 usec, L: 0
0320 15:27:24.930395 (+ 161us) write_op.cc:270] Start()
0320 15:27:24.930431 (+ 36us) write_op.cc:276] Timestamp: P: 1774020441550808 usec, L: 0
0320 15:27:24.930625 (+ 194us) write_op.cc:270] Start()
0320 15:27:24.930648 (+ 23us) write_op.cc:276] Timestamp: P: 1774020441606390 usec, L: 0
0320 15:27:24.930768 (+ 120us) write_op.cc:270] Start()
0320 15:27:24.930785 (+ 17us) write_op.cc:276] Timestamp: P: 1774020441620461 usec, L: 0
0320 15:27:24.930889 (+ 104us) write_op.cc:270] Start()
0320 15:27:24.930907 (+ 18us) write_op.cc:276] Timestamp: P: 1774020441621146 usec, L: 0
0320 15:27:24.931032 (+ 125us) write_op.cc:270] Start()
0320 15:27:24.931049 (+ 17us) write_op.cc:276] Timestamp: P: 1774020441621796 usec, L: 0
0320 15:27:24.931187 (+ 138us) write_op.cc:270] Start()
0320 15:27:24.931206 (+ 19us)
I20260320 15:27:25.897616 28730 ts_tablet_manager.cc:1916] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.897943 28054 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:25.898183 28054 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_e081f8a109224b479a6e7aec2d18356b [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 1 got 0
I20260320 15:27:25.900540 28729 raft_consensus.cc:2243] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.902575 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 442ca186c9d4449dade39271dec5c608
I20260320 15:27:25.903218 28053 catalog_manager.cc:6741] ReplaceTablet: tablet 442ca186c9d4449dade39271dec5c608 deleted and replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5
W20260320 15:27:25.905149 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 889d620c311a42039c22af53ad0802f2 consensus state has no leader
I20260320 15:27:25.906015 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 533105ad6e054fe3a4db32855bc688bd
I20260320 15:27:25.906634 28729 pending_rounds.cc:70] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:25.906744 28729 pending_rounds.cc:77] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387740741345280 op_type: NO_OP noop_request { }
I20260320 15:27:25.907940 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.907953 28729 raft_consensus.cc:2889] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:25.908107 28729 raft_consensus.cc:2272] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.909510 28411 tablet_service.cc:1511] Processing CreateTablet for tablet 23eda54577404cc6900cb3ae348604d4 (DEFAULT_TABLE table=default.Table_36123e5c9fed4e51bfab33f47ed1d82b [id=bf25edfb9f0e4924aadf3e85e57fab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.910158 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6b6165eab6b0442fa9d94ce577c1b43c (table default.Table_1e7d8c5d8d364cadbf3a1e3e1a5bf50c [id=56dc54e66dbb4e68a56a4e0f01c2ae47]) successfully deleted
I20260320 15:27:25.913606 28735 tablet_replica.cc:333] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:25.913985 28735 raft_consensus.cc:2243] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:25.915239 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23eda54577404cc6900cb3ae348604d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.916045 28735 raft_consensus.cc:2272] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:25.917142 28735 ts_tablet_manager.cc:1916] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.918954 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.918493 28736 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.920406 28736 ts_tablet_manager.cc:1403] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.108s user 0.002s sys 0.006s
I20260320 15:27:25.921450 28736 raft_consensus.cc:359] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.921669 28736 raft_consensus.cc:385] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.921698 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:25.921762 28736 raft_consensus.cc:740] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.922046 28736 consensus_queue.cc:260] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.923326 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet c3bc4dd47817490d88725127b456cc70 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:25.923799 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:25.924712 28729 ts_tablet_manager.cc:1916] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:25.925597 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.925807 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 60d0b36f462945cbbca7e389edf494d3 (DEFAULT_TABLE table=default.Table_c2a40da69e9d4735b3dd5e71d419b00b [id=cfa06a2552dd47ca947741764a0bf762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.925956 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet c3bc4dd47817490d88725127b456cc70
I20260320 15:27:25.926353 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d0b36f462945cbbca7e389edf494d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.928050 28736 ts_tablet_manager.cc:1434] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:25.927469 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 07af28f257d745a1bb4173aad09045d9 (DEFAULT_TABLE table=default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.928098 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ac4c7dc6859048ce956927201df44a86
I20260320 15:27:25.930331 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e081f8a109224b479a6e7aec2d18356b" } new_table_name: "default.Table_f355c2a83bf94395b0e1234d65483b00" modify_external_catalogs: true
I20260320 15:27:25.932984 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_ba23f221189b481191026b461623223b" } new_table_name: "default.Table_5d8b2e640491465da035984a69b16fcc" modify_external_catalogs: true
I20260320 15:27:25.933116 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af28f257d745a1bb4173aad09045d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.935674 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.936105 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d7023c991b824e1a87f6fc910db487bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.936617 28736 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:25.936777 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7023c991b824e1a87f6fc910db487bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.937151 29146 raft_consensus.cc:493] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:25.937382 29146 raft_consensus.cc:515] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.937829 28734 tablet_bootstrap.cc:654] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.938099 29146 leader_election.cc:290] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:25.938299 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_69342bafde2c42ab8c8eaedb0f28dfb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.938987 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.939755 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:25.941107 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:25.941107 28240 leader_election.cc:343] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.941411 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.941720 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_69865560dad94f49bc2c800c90e17cbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:25.942746 28736 tablet_bootstrap.cc:654] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.942719 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:25.942950 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69865560dad94f49bc2c800c90e17cbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.944161 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.944859 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.945466 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:25.945957 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.946046 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_fb580f5b6e1141e1a236805952cf1214"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.947029 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb580f5b6e1141e1a236805952cf1214 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.949121 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:25.949216 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_5803227c57ec4896947ecce23135454f" } new_table_name: "default.Table_a9395e2f7cc54d5eac9e394cdb342ad9" modify_external_catalogs: true
I20260320 15:27:25.952776 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_6673a2f894aa425bb079ad34bf6940b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:25.953681 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6673a2f894aa425bb079ad34bf6940b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:25.960256 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:25.960414 28736 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:25.960620 28736 ts_tablet_manager.cc:1403] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20260320 15:27:25.961227 28554 tablet_service.cc:1511] Processing CreateTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f (DEFAULT_TABLE table=default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.961740 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd8f5ccbd35418684fd27459c7a6b9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.962141 28736 raft_consensus.cc:359] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.962365 28736 raft_consensus.cc:385] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:25.962436 28736 raft_consensus.cc:740] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:25.939093 29176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "07e8bb8819eb4cb4892e0cba1f758892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:25.962841 29176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:25.962821 28736 consensus_queue.cc:260] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:25.963589 28736 ts_tablet_manager.cc:1434] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:25.964010 28736 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:25.966279 28736 tablet_bootstrap.cc:654] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:25.970659 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet da957aed322f46acaf5876c567c2ff94 (table default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]) successfully deleted
I20260320 15:27:25.980410 28729 ts_tablet_manager.cc:1929] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:25.980579 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 60d0b36f462945cbbca7e389edf494d3 (DEFAULT_TABLE table=default.Table_c2a40da69e9d4735b3dd5e71d419b00b [id=cfa06a2552dd47ca947741764a0bf762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.980731 28729 log.cc:1199] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/203c619513a146f49a4825962f169f45
I20260320 15:27:25.981144 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60d0b36f462945cbbca7e389edf494d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.981601 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 23eda54577404cc6900cb3ae348604d4 (DEFAULT_TABLE table=default.Table_36123e5c9fed4e51bfab33f47ed1d82b [id=bf25edfb9f0e4924aadf3e85e57fab77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:25.982339 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23eda54577404cc6900cb3ae348604d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:25.983700 28564 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.983983 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.984175 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.984382 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.984643 28549 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:25.984870 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:25.997745 28239 leader_election.cc:343] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:25.997941 28239 leader_election.cc:304] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:25.998595 29146 raft_consensus.cc:2749] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.000442 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 07af28f257d745a1bb4173aad09045d9 (DEFAULT_TABLE table=default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.002336 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af28f257d745a1bb4173aad09045d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.002600 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 54266dd376164673bffe829dd515b83d (DEFAULT_TABLE table=default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.002578 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.003014 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54266dd376164673bffe829dd515b83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.003057 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771bd6a3f0e47d99a0191eae25cb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.008188 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.008580 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 54266dd376164673bffe829dd515b83d (DEFAULT_TABLE table=default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.008754 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771bd6a3f0e47d99a0191eae25cb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.008993 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54266dd376164673bffe829dd515b83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.009356 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 07af28f257d745a1bb4173aad09045d9 (DEFAULT_TABLE table=default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.009863 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07af28f257d745a1bb4173aad09045d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.023334 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 54266dd376164673bffe829dd515b83d (DEFAULT_TABLE table=default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.024013 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54266dd376164673bffe829dd515b83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.026937 28411 tablet_service.cc:1511] Processing CreateTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.027557 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771bd6a3f0e47d99a0191eae25cb2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.035162 28277 tablet_service.cc:1511] Processing CreateTablet for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (DEFAULT_TABLE table=default.Table_a5ca3ae6cd61430db4e2c4112c011e95 [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.035835 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78ab2f7e0d7413ea9049dba9cbd8633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.039006 29146 raft_consensus.cc:493] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.039378 29146 raft_consensus.cc:515] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.040179 29146 leader_election.cc:290] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.049057 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:26.049573 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_dfd551c35b134e27a2424da18f1a5bd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.050215 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfd551c35b134e27a2424da18f1a5bd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:26.051968 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7a4b5a658cd84b04b50b54973c065889 consensus state has no leader
I20260320 15:27:26.052395 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a15c29714da04984898578d391977a08" } new_table_name: "default.Table_12977445848d4334ab141542db268e90" modify_external_catalogs: true
W20260320 15:27:26.054324 28240 leader_election.cc:343] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.055217 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_094bdcada4784500a661a56536bb221b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.055827 29146 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:26.055868 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_094bdcada4784500a661a56536bb221b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.056051 29146 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.057065 29146 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:26.058247 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa5618796f1a4ff3ab4c1209dad0aede consensus state has no leader
I20260320 15:27:26.058221 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:26.058681 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e39f3c720ae24bf5978338b793412e12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.059034 28240 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:26.059278 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e39f3c720ae24bf5978338b793412e12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.060322 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet 9176e734212849aab8d99473de02cc79 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:26.062592 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b4040a0214a46b580b2ac2de59243ce consensus state has no leader
I20260320 15:27:26.062954 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet 45a0e8e2ce7a454ebcab4e59a4de68dd from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.074039 28054 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed] alter complete (version 3)
I20260320 15:27:26.092263 28729 ts_tablet_manager.cc:1950] T 203c619513a146f49a4825962f169f45 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:26.109599 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e847499477ab4a5ca57d41599f7605dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.110039 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_69865560dad94f49bc2c800c90e17cbf" } new_table_name: "default.Table_78c6dd62d8834a39afa05aac3371a7c0" modify_external_catalogs: true
I20260320 15:27:26.109869 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:26.110443 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e847499477ab4a5ca57d41599f7605dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.116775 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 262) took 1668 ms. Trace:
I20260320 15:27:26.117081 28729 rpcz_store.cc:276] 0320 15:27:24.448358 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.449512 (+ 1154us) service_pool.cc:225] Handling call
0320 15:27:25.872119 (+1422607us) ts_tablet_manager.cc:1151] Deleting tablet 203c619513a146f49a4825962f169f45
0320 15:27:25.872134 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.953707 (+ 81573us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:25.980256 (+ 26549us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.116656 (+136400us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1420379}
I20260320 15:27:26.118150 28729 tablet_replica.cc:333] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:26.118585 28729 raft_consensus.cc:2243] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.118950 28729 raft_consensus.cc:2272] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.125180 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 8241a342054240d2ac62b729cc9c9a62 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.125707 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8241a342054240d2ac62b729cc9c9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.126112 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 203c619513a146f49a4825962f169f45 (table default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]) successfully deleted
I20260320 15:27:26.128564 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9f87c6321cdc405ab18f525e320707f2" } modify_external_catalogs: true
I20260320 15:27:26.128799 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9f87c6321cdc405ab18f525e320707f2" } modify_external_catalogs: true
I20260320 15:27:26.124027 28424 tablet_service.cc:1511] Processing CreateTablet for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (DEFAULT_TABLE table=default.Table_a5ca3ae6cd61430db4e2c4112c011e95 [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.140091 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78ab2f7e0d7413ea9049dba9cbd8633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.140694 29134 raft_consensus.cc:493] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.141036 29134 raft_consensus.cc:515] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.124312 28734 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.125456 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 8241a342054240d2ac62b729cc9c9a62 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.147840 28734 ts_tablet_manager.cc:1403] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.272s user 0.004s sys 0.003s
I20260320 15:27:26.148167 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8241a342054240d2ac62b729cc9c9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.148916 28734 raft_consensus.cc:359] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.149375 28734 raft_consensus.cc:385] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.149533 28734 raft_consensus.cc:740] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.153481 29122 tablet.cc:1723] T 0d21b0e8db1a424ea329b8f9cd03ab5b P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:26.162010 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.165867 28419 tablet_service.cc:1511] Processing CreateTablet for tablet fc009401bb3d440f9d36e080efef1204 (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:26.166326 28524 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:26.168771 28736 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:26.169036 28736 ts_tablet_manager.cc:1403] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.205s user 0.041s sys 0.072s
I20260320 15:27:26.170164 28736 raft_consensus.cc:359] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.170439 28736 raft_consensus.cc:385] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.171082 28736 raft_consensus.cc:740] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.171466 28736 consensus_queue.cc:260] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.172192 28736 ts_tablet_manager.cc:1434] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:26.173869 28735 ts_tablet_manager.cc:1929] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:26.174144 28735 log.cc:1199] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/28c1dcc311a14b33a289ae2b8a485d2b
I20260320 15:27:26.176069 29134 leader_election.cc:290] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.174934 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.177974 28735 ts_tablet_manager.cc:1950] T 28c1dcc311a14b33a289ae2b8a485d2b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:26.178277 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28c453eaf49c4171b0f93d3b26690d83 has no consensus state
W20260320 15:27:26.178529 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5a20d7f22cfa403184e73b24d565b60a consensus state has no leader
I20260320 15:27:26.178889 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3d0dcd71707248d49f09c2356ee00f6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.179018 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 286) took 1209 ms. Trace:
I20260320 15:27:26.179126 28735 rpcz_store.cc:276] 0320 15:27:24.969030 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.969103 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:25.913527 (+944424us) ts_tablet_manager.cc:1151] Deleting tablet 28c1dcc311a14b33a289ae2b8a485d2b
0320 15:27:25.913537 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.157330 (+243793us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.173729 (+ 16399us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.178956 (+ 5227us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":944174}
I20260320 15:27:26.179654 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0703f70973474e499d7916aa2a63f195" } modify_external_catalogs: true
I20260320 15:27:26.179824 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_0703f70973474e499d7916aa2a63f195" } modify_external_catalogs: true
I20260320 15:27:26.180083 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_17c9ae5b739344f4ab272260023b6a74" } modify_external_catalogs: true
I20260320 15:27:26.180284 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_17c9ae5b739344f4ab272260023b6a74" } modify_external_catalogs: true
I20260320 15:27:26.180362 28046 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a861f30c768b4ee4988c1f3881eefa19" } modify_external_catalogs: true
I20260320 15:27:26.180531 28046 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a861f30c768b4ee4988c1f3881eefa19" } modify_external_catalogs: true
W20260320 15:27:26.181633 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d0dcd71707248d49f09c2356ee00f6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.183066 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c" } modify_external_catalogs: true
I20260320 15:27:26.183286 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c" } modify_external_catalogs: true
I20260320 15:27:26.184487 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 89425da569794a8fa972817023bf853d
I20260320 15:27:26.186921 28735 tablet_replica.cc:333] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:26.187461 28736 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:26.192591 28736 tablet_bootstrap.cc:654] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.193409 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.193820 28730 ts_tablet_manager.cc:1929] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.193964 28730 log.cc:1199] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/545c8fd2d4c845adba6dbf42dfddba3e
W20260320 15:27:26.192978 28524 leader_election.cc:343] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:26.195277 28730 ts_tablet_manager.cc:1950] T 545c8fd2d4c845adba6dbf42dfddba3e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:26.195648 28524 leader_election.cc:304] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.195678 28735 raft_consensus.cc:2243] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.196005 28735 raft_consensus.cc:2272] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.196295 29134 raft_consensus.cc:2749] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.196962 28735 ts_tablet_manager.cc:1916] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.197404 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 28c1dcc311a14b33a289ae2b8a485d2b (table default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]) successfully deleted
I20260320 15:27:26.198181 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 9176e734212849aab8d99473de02cc79
I20260320 15:27:26.198835 28047 catalog_manager.cc:6741] ReplaceTablet: tablet 9176e734212849aab8d99473de02cc79 deleted and replaced by tablet dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:26.207054 28054 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_4495058468dd4441be0cbc0668f6bb8b [id=399646419f3b4fd28458144f43bd4553] alter complete (version 3)
I20260320 15:27:26.204174 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.214901 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 248) took 1828 ms. Trace:
I20260320 15:27:26.216114 28730 rpcz_store.cc:276] 0320 15:27:24.386763 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.386905 (+ 142us) service_pool.cc:225] Handling call
0320 15:27:25.886427 (+1499522us) ts_tablet_manager.cc:1151] Deleting tablet 545c8fd2d4c845adba6dbf42dfddba3e
0320 15:27:25.886437 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:25.977562 (+ 91125us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.192644 (+215082us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.214856 (+ 22212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1499249}
I20260320 15:27:26.216696 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 249) took 1829 ms. Trace:
I20260320 15:27:26.217139 28730 rpcz_store.cc:276] 0320 15:27:24.387399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.387761 (+ 362us) service_pool.cc:225] Handling call
0320 15:27:26.216520 (+1828759us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:26.216530 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.216660 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1828481}
I20260320 15:27:26.217687 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 250) took 1634 ms. Trace:
I20260320 15:27:26.221130 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:26.224001 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:26.225539 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:26.226598 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 07af28f257d745a1bb4173aad09045d9
I20260320 15:27:26.230582 28729 ts_tablet_manager.cc:1916] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.230850 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.231161 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc009401bb3d440f9d36e080efef1204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.231806 28277 tablet_service.cc:1511] Processing CreateTablet for tablet fc009401bb3d440f9d36e080efef1204 (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.232357 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc009401bb3d440f9d36e080efef1204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.233016 28734 consensus_queue.cc:260] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.234503 28730 rpcz_store.cc:276] 0320 15:27:24.583580 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.599181 (+ 15601us) service_pool.cc:225] Handling call
0320 15:27:26.217543 (+1618362us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:26.217553 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.217653 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1609053}
I20260320 15:27:26.235359 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 256) took 1463 ms. Trace:
I20260320 15:27:26.235827 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.236392 28730 rpcz_store.cc:276] 0320 15:27:24.772226 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.772306 (+ 80us) service_pool.cc:225] Handling call
0320 15:27:26.235042 (+1462736us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:26.235058 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.235193 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1462414}
I20260320 15:27:26.238695 28730 tablet_replica.cc:333] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:26.240901 28730 raft_consensus.cc:2243] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:26.241333 28730 pending_rounds.cc:70] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:26.241771 28730 pending_rounds.cc:77] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387740899573760 op_type: NO_OP noop_request { }
I20260320 15:27:26.241987 28730 raft_consensus.cc:2889] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.242199 28730 raft_consensus.cc:2272] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.242544 29146 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.242756 29146 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.243443 29146 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.245916 29134 consensus_queue.cc:1048] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:26.251225 28734 ts_tablet_manager.cc:1434] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.088s user 0.001s sys 0.002s
I20260320 15:27:26.246454 28277 tablet_service.cc:1511] Processing CreateTablet for tablet e0e7ae1ff302428abf6cc592bbca0aa9 (DEFAULT_TABLE table=default.Table_2827438ad3b74f42968ceb52cf633bff [id=f1748199619b4638917812d77feed8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.252619 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4b5a658cd84b04b50b54973c065889 (DEFAULT_TABLE table=default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.253589 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4b5a658cd84b04b50b54973c065889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.253569 28549 tablet_service.cc:1511] Processing CreateTablet for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (DEFAULT_TABLE table=default.Table_a5ca3ae6cd61430db4e2c4112c011e95 [id=03aa8e7e0e044e31b29509e7ed1ff617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.253831 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 0acad11980f842e7b3a09216241e493d (DEFAULT_TABLE table=default.Table_1bbba63ee33c407f8d02e8f63df7245e [id=e703fc2e6e2345138d5a6852097ade20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.254215 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acad11980f842e7b3a09216241e493d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.255646 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 0acad11980f842e7b3a09216241e493d (DEFAULT_TABLE table=default.Table_1bbba63ee33c407f8d02e8f63df7245e [id=e703fc2e6e2345138d5a6852097ade20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.256089 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acad11980f842e7b3a09216241e493d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:26.261057 28383 consensus_peers.cc:597] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da957aed322f46acaf5876c567c2ff94. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.262042 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0e7ae1ff302428abf6cc592bbca0aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.263063 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d78ab2f7e0d7413ea9049dba9cbd8633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.263000 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 01078353a3a343a3a1231245a391cf8e (DEFAULT_TABLE table=default.Table_42657ecb01cc40af923263a4556db5df [id=b9bf566229c34b7e84767938e2f605ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.263442 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01078353a3a343a3a1231245a391cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.264207 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d1db9963c4645ae9aab72b4b22d6938 with delete_type TABLET_DATA_DELETED (replaced by tablet d56de00174db4e2d8b1c7215038a04aa) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.266371 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4b5a658cd84b04b50b54973c065889 (DEFAULT_TABLE table=default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.266866 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4b5a658cd84b04b50b54973c065889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.269455 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ada1f644b754fbe8b3790b46ec8cdf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.272611 28730 ts_tablet_manager.cc:1916] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.271970 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.275689 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.277215 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.278785 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.279510 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.281370 28734 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.284096 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.288373 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.288424 28426 tablet_service.cc:1511] Processing CreateTablet for tablet e0e7ae1ff302428abf6cc592bbca0aa9 (DEFAULT_TABLE table=default.Table_2827438ad3b74f42968ceb52cf633bff [id=f1748199619b4638917812d77feed8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.290112 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:26.292110 28240 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:26.292200 28524 consensus_peers.cc:597] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 203c619513a146f49a4825962f169f45. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.292707 28917 raft_consensus.cc:493] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.292827 28735 ts_tablet_manager.cc:1929] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:26.292949 28735 log.cc:1199] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:26.292892 28917 raft_consensus.cc:515] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.293355 28735 ts_tablet_manager.cc:1950] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:26.294229 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 287) took 1325 ms. Trace:
I20260320 15:27:26.294335 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.294361 28735 rpcz_store.cc:276] 0320 15:27:24.969142 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.969386 (+ 244us) service_pool.cc:225] Handling call
0320 15:27:26.186799 (+1217413us) ts_tablet_manager.cc:1151] Deleting tablet 5ada1f644b754fbe8b3790b46ec8cdf4
0320 15:27:26.186816 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.289090 (+102274us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.292657 (+ 3567us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.294183 (+ 1526us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1217229}
I20260320 15:27:26.294791 28735 tablet_replica.cc:333] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:26.295077 28735 raft_consensus.cc:2243] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.295339 28735 raft_consensus.cc:2272] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.295504 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0e7ae1ff302428abf6cc592bbca0aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.296342 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.297539 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.301843 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5ada1f644b754fbe8b3790b46ec8cdf4 (table default.Table_0b295d872d094ac3bca1ea9effaf3081 [id=5de6c807af97457ea392b60abf33a34b]) successfully deleted
I20260320 15:27:26.302062 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec63fb080545de838803033f204fab (DEFAULT_TABLE table=default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.302532 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec63fb080545de838803033f204fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.306198 28270 tablet_service.cc:1511] Processing CreateTablet for tablet ec6961acec774d2f9161f057f64b3fdf (DEFAULT_TABLE table=default.Table_089d0cb58bd1495cbe761dae3b90b562 [id=8fe18e7035b544b3bd5b5ef3da3bd504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.306818 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6961acec774d2f9161f057f64b3fdf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:26.307823 28383 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
W20260320 15:27:26.311924 28524 consensus_peers.cc:597] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 203c619513a146f49a4825962f169f45. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.312084 28272 tablet_service.cc:1511] Processing CreateTablet for tablet e5479578a7cd470a9a4851d252a9e93e (DEFAULT_TABLE table=default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 [id=0a1df577ad734c34b6cfe3cba679ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.312615 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5479578a7cd470a9a4851d252a9e93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.313861 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.314172 28567 tablet_service.cc:1511] Processing CreateTablet for tablet fc009401bb3d440f9d36e080efef1204 (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.316826 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc009401bb3d440f9d36e080efef1204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.319218 28562 tablet_service.cc:1511] Processing CreateTablet for tablet e0e7ae1ff302428abf6cc592bbca0aa9 (DEFAULT_TABLE table=default.Table_2827438ad3b74f42968ceb52cf633bff [id=f1748199619b4638917812d77feed8f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.319918 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0e7ae1ff302428abf6cc592bbca0aa9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:26.320348 28382 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:26.320562 28917 leader_election.cc:290] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.321048 28735 ts_tablet_manager.cc:1916] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.321465 28382 leader_election.cc:304] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.321712 28418 tablet_service.cc:1511] Processing CreateTablet for tablet ec6961acec774d2f9161f057f64b3fdf (DEFAULT_TABLE table=default.Table_089d0cb58bd1495cbe761dae3b90b562 [id=8fe18e7035b544b3bd5b5ef3da3bd504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.322643 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 01078353a3a343a3a1231245a391cf8e (DEFAULT_TABLE table=default.Table_42657ecb01cc40af923263a4556db5df [id=b9bf566229c34b7e84767938e2f605ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.323446 29156 raft_consensus.cc:2749] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.324669 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6961acec774d2f9161f057f64b3fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.324877 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 8241a342054240d2ac62b729cc9c9a62 (DEFAULT_TABLE table=default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.326576 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4b5a658cd84b04b50b54973c065889 (DEFAULT_TABLE table=default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.329083 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 0acad11980f842e7b3a09216241e493d (DEFAULT_TABLE table=default.Table_1bbba63ee33c407f8d02e8f63df7245e [id=e703fc2e6e2345138d5a6852097ade20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.330842 28421 tablet_service.cc:1511] Processing CreateTablet for tablet e5479578a7cd470a9a4851d252a9e93e (DEFAULT_TABLE table=default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 [id=0a1df577ad734c34b6cfe3cba679ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.332059 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8241a342054240d2ac62b729cc9c9a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.332465 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0acad11980f842e7b3a09216241e493d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.332566 28734 tablet_bootstrap.cc:654] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.332396 29156 consensus_queue.cc:1048] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260320 15:27:26.333415 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4b5a658cd84b04b50b54973c065889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.335296 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.337073 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5479578a7cd470a9a4851d252a9e93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.338459 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec63fb080545de838803033f204fab (DEFAULT_TABLE table=default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.338944 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec63fb080545de838803033f204fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.339133 28549 tablet_service.cc:1511] Processing CreateTablet for tablet 01078353a3a343a3a1231245a391cf8e (DEFAULT_TABLE table=default.Table_42657ecb01cc40af923263a4556db5df [id=b9bf566229c34b7e84767938e2f605ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.340965 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.341298 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.341676 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.342132 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 889d620c311a42039c22af53ad0802f2 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.343011 28563 tablet_service.cc:1511] Processing CreateTablet for tablet ec6961acec774d2f9161f057f64b3fdf (DEFAULT_TABLE table=default.Table_089d0cb58bd1495cbe761dae3b90b562 [id=8fe18e7035b544b3bd5b5ef3da3bd504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.343796 28551 tablet_service.cc:1511] Processing CreateTablet for tablet e5479578a7cd470a9a4851d252a9e93e (DEFAULT_TABLE table=default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 [id=0a1df577ad734c34b6cfe3cba679ed7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.344789 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec63fb080545de838803033f204fab (DEFAULT_TABLE table=default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.345476 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 545c8fd2d4c845adba6dbf42dfddba3e (table default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]) successfully deleted
I20260320 15:27:26.320418 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 889d620c311a42039c22af53ad0802f2 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:26.344030 28239 leader_election.cc:343] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.345988 28239 leader_election.cc:304] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:26.347739 28239 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.347901 28239 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.347981 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_047ea6293f154736b2bff4875bc8cd43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.348491 29146 raft_consensus.cc:2749] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.348547 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2926e23005b34db8a1d7e340b1a1e55e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.348743 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_047ea6293f154736b2bff4875bc8cd43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.348752 29146 raft_consensus.cc:2749] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:26.349085 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2926e23005b34db8a1d7e340b1a1e55e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.349119 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a341aa1a6fbe4dedb77cedb3339ea59d" } new_table_name: "default.Table_e6270dc01722414ab060a44ccc4125fa" modify_external_catalogs: true
I20260320 15:27:26.353506 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_00ee6904094c4b5abf6e50334f698c40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.354221 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_00ee6904094c4b5abf6e50334f698c40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.357971 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 889d620c311a42039c22af53ad0802f2 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:26.359901 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9ee2dbc7d4f343f2b58a9fee8543896a" } new_table_name: "default.Table_1840a78bd7454ea2b4f4fe40a849b311" modify_external_catalogs: true
I20260320 15:27:26.362607 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.368649 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ce843e88d7f24053b0be457f201e11ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.369316 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce843e88d7f24053b0be457f201e11ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.369570 28734 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.369781 28734 ts_tablet_manager.cc:1403] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.089s user 0.004s sys 0.007s
I20260320 15:27:26.373284 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_003b2dd0dbb34453bba65abed2b789e9" } new_table_name: "default.Table_e9d83c80d8c64b19af48bda73dff8d8c" modify_external_catalogs: true
I20260320 15:27:26.373580 29156 consensus_queue.cc:1048] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:26.375216 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_536e57665fa14e0e915512c1a27ab8fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.376202 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_536e57665fa14e0e915512c1a27ab8fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.382824 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889d620c311a42039c22af53ad0802f2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:26.383561 28050 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aafd1e123cd043e7828140d2443e047f consensus state has no leader
I20260320 15:27:26.385389 28734 raft_consensus.cc:359] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.389078 28734 raft_consensus.cc:385] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.391085 28734 raft_consensus.cc:740] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.391448 28734 consensus_queue.cc:260] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.394083 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b4cad900411e4c9686701aa99206dd23 consensus state has no leader
W20260320 15:27:26.396571 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f5beaa1bc1e4465be34c8e86f131559 consensus state has no leader
I20260320 15:27:26.402263 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01078353a3a343a3a1231245a391cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.402727 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec63fb080545de838803033f204fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.403611 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5479578a7cd470a9a4851d252a9e93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.404263 28729 ts_tablet_manager.cc:1929] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:26.404453 28729 log.cc:1199] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/d57566e7907c4783aafce347e46b89e3
W20260320 15:27:26.406453 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a072859f39c4c28ae65b0e9afa79d66 already in progress: creating tablet
W20260320 15:27:26.407311 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea6ff59d2ffe4d579e6db2375eb11414 already in progress: creating tablet
I20260320 15:27:26.407696 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec6961acec774d2f9161f057f64b3fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.407891 28734 ts_tablet_manager.cc:1434] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.035s user 0.003s sys 0.000s
W20260320 15:27:26.408437 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet edb7b53bb7444a39b471218c6039542f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb7b53bb7444a39b471218c6039542f already in progress: creating tablet
I20260320 15:27:26.408672 29156 raft_consensus.cc:493] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.409416 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.409976 28734 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.414793 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889d620c311a42039c22af53ad0802f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.416785 29156 raft_consensus.cc:515] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.418231 28734 tablet_bootstrap.cc:654] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.419003 28729 ts_tablet_manager.cc:1950] T d57566e7907c4783aafce347e46b89e3 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:26.420199 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 263) took 1971 ms. Trace:
I20260320 15:27:26.421744 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:26.425765 28382 leader_election.cc:343] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.423166 29156 leader_election.cc:290] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.433526 28729 rpcz_store.cc:276] 0320 15:27:24.448546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.449687 (+ 1141us) service_pool.cc:225] Handling call
0320 15:27:26.117967 (+1668280us) ts_tablet_manager.cc:1151] Deleting tablet d57566e7907c4783aafce347e46b89e3
0320 15:27:26.117985 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.317861 (+199876us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.393887 (+ 76026us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.420148 (+ 26261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1666130}
I20260320 15:27:26.436143 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 889d620c311a42039c22af53ad0802f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.439131 28729 tablet_replica.cc:333] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:26.446920 28734 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.447145 28734 ts_tablet_manager.cc:1403] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.003s
I20260320 15:27:26.448587 28734 raft_consensus.cc:359] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.449358 28734 raft_consensus.cc:385] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.449721 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01078353a3a343a3a1231245a391cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:26.449530 28734 raft_consensus.cc:740] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:26.450040 28382 consensus_peers.cc:597] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da957aed322f46acaf5876c567c2ff94. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.450345 28734 consensus_queue.cc:260] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.452116 28734 ts_tablet_manager.cc:1434] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260320 15:27:26.453735 28734 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:26.459707 28733 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.013s sys 0.017s
W20260320 15:27:26.459995 28733 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.588s user 0.013s sys 0.017s
I20260320 15:27:26.468554 28734 tablet_bootstrap.cc:654] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.472657 28729 raft_consensus.cc:2243] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:26.473201 28729 pending_rounds.cc:70] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:26.473480 28735 ts_tablet_manager.cc:1929] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:26.473670 28735 log.cc:1199] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/d57566e7907c4783aafce347e46b89e3
I20260320 15:27:26.473814 28729 pending_rounds.cc:77] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387743713869824 op_type: NO_OP noop_request { }
I20260320 15:27:26.474016 28729 raft_consensus.cc:2889] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.474133 28735 ts_tablet_manager.cc:1950] T d57566e7907c4783aafce347e46b89e3 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:26.474501 28729 raft_consensus.cc:2272] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.475078 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 288) took 1505 ms. Trace:
I20260320 15:27:26.475201 28735 rpcz_store.cc:276] 0320 15:27:24.969334 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.969455 (+ 121us) service_pool.cc:225] Handling call
0320 15:27:26.294686 (+1325231us) ts_tablet_manager.cc:1151] Deleting tablet d57566e7907c4783aafce347e46b89e3
0320 15:27:26.294699 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.333858 (+ 39159us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.473330 (+139472us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.474989 (+ 1659us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1324958}
W20260320 15:27:26.482046 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 134602us
I20260320 15:27:26.488708 28734 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.488963 28734 ts_tablet_manager.cc:1403] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.007s
I20260320 15:27:26.489914 28729 ts_tablet_manager.cc:1916] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.490031 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:26.490307 28300 raft_consensus.cc:2468] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:26.491637 28239 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.492069 28239 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.492524 29188 raft_consensus.cc:2749] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.490039 28734 raft_consensus.cc:359] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.493414 28734 raft_consensus.cc:385] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.493420 29156 consensus_queue.cc:1048] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:26.493613 28734 raft_consensus.cc:740] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.493903 28734 consensus_queue.cc:260] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.496551 28734 ts_tablet_manager.cc:1434] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260320 15:27:26.497058 28734 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.497154 28383 leader_election.cc:304] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.497807 29156 raft_consensus.cc:2804] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.498132 29156 raft_consensus.cc:493] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.498399 29156 raft_consensus.cc:3060] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.499361 28734 tablet_bootstrap.cc:654] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.502048 29156 raft_consensus.cc:515] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.504597 29188 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.504967 29188 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.505764 29188 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.506693 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:26.507225 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:26.507908 28240 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.508157 29156 leader_election.cc:290] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:26.508562 28382 leader_election.cc:343] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.509449 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.511257 29156 raft_consensus.cc:493] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.512142 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet d57566e7907c4783aafce347e46b89e3 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]) successfully deleted
I20260320 15:27:26.512760 29156 raft_consensus.cc:515] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.513377 28239 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:26.515913 28524 consensus_peers.cc:597] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0e8f37a490a4fdfbc2a3dc56f6da297. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.517746 28733 tablet_bootstrap.cc:492] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:26.518110 28733 ts_tablet_manager.cc:1403] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.677s user 0.015s sys 0.019s
I20260320 15:27:26.519062 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_62a50c277a2f45bd926514ab39b3b7e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.519091 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_fda3998f623540e6beca0a6f3455b0b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.519858 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62a50c277a2f45bd926514ab39b3b7e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:26.532104 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fda3998f623540e6beca0a6f3455b0b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.513427 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:26.534575 28300 raft_consensus.cc:3060] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.535285 28052 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:26.537639 29156 leader_election.cc:290] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.537860 28300 raft_consensus.cc:2468] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:26.538400 28239 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.541641 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_1840a78bd7454ea2b4f4fe40a849b311" } new_table_name: "default.Table_134b3b5c79e344f2bf710b12615479be" modify_external_catalogs: true
I20260320 15:27:26.537385 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:26.542960 28382 leader_election.cc:343] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:26.565596 28524 consensus_peers.cc:597] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28c1dcc311a14b33a289ae2b8a485d2b. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.569015 29199 raft_consensus.cc:493] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.570848 28733 raft_consensus.cc:359] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.572358 28049 master_service.cc:954] ReplaceTablet: received request to replace tablet fa5618796f1a4ff3ab4c1209dad0aede from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.573545 29188 raft_consensus.cc:2749] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:26.574172 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 60772.3us
I20260320 15:27:26.577216 29199 raft_consensus.cc:515] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.569788 28735 tablet_replica.cc:333] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:26.578334 28735 raft_consensus.cc:2243] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:26.578691 28735 pending_rounds.cc:70] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:26.578792 28735 pending_rounds.cc:77] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387745620959232 op_type: NO_OP noop_request { }
I20260320 15:27:26.578964 28735 raft_consensus.cc:2889] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.579092 28735 raft_consensus.cc:2272] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.585603 29199 leader_election.cc:290] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.586370 28383 leader_election.cc:304] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.587847 29156 raft_consensus.cc:2804] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:26.588141 29156 raft_consensus.cc:697] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:26.588636 29156 consensus_queue.cc:237] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.596329 28734 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.596604 28734 ts_tablet_manager.cc:1403] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.100s user 0.012s sys 0.028s
I20260320 15:27:26.597792 28734 raft_consensus.cc:359] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.598038 28734 raft_consensus.cc:385] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.598116 28734 raft_consensus.cc:740] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.598872 28734 consensus_queue.cc:260] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.599505 28734 ts_tablet_manager.cc:1434] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260320 15:27:26.599907 28734 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.602156 28734 tablet_bootstrap.cc:654] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:26.611285 28383 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:26.612151 28734 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.612334 28734 ts_tablet_manager.cc:1403] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.002s
I20260320 15:27:26.612445 28047 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4df55a5738d24d4292a69c7e5550865b" } modify_external_catalogs: true
I20260320 15:27:26.613823 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:26.614126 28300 raft_consensus.cc:2468] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:26.615641 28734 raft_consensus.cc:359] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.615867 28734 raft_consensus.cc:385] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.615969 28734 raft_consensus.cc:740] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.616220 28734 consensus_queue.cc:260] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.616849 28734 ts_tablet_manager.cc:1434] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:26.617091 28383 leader_election.cc:304] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.617396 28917 raft_consensus.cc:2804] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.617507 28917 raft_consensus.cc:493] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.617650 28917 raft_consensus.cc:3060] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.619428 28734 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.620165 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_98404b0cbf764bed9e21a83afa8a7dcd" } new_table_name: "default.Table_b23b6416f43846bab7b604e6b7935bad" modify_external_catalogs: true
I20260320 15:27:26.621588 28734 tablet_bootstrap.cc:654] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.624078 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_eb8fe7340adc409c98903887ac581604"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.624819 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb8fe7340adc409c98903887ac581604 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.626221 28047 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_4df55a5738d24d4292a69c7e5550865b" } modify_external_catalogs: true
I20260320 15:27:26.627409 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet d57566e7907c4783aafce347e46b89e3 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]) successfully deleted
I20260320 15:27:26.627734 28053 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a5952001c570473c902abb9d529c60f1" } modify_external_catalogs: true
I20260320 15:27:26.627945 28053 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a5952001c570473c902abb9d529c60f1" } modify_external_catalogs: true
I20260320 15:27:26.628075 28734 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.628257 28734 ts_tablet_manager.cc:1403] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260320 15:27:26.629060 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_ef4a998b01e2437a8edcf95a89e5a314"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.629194 28734 raft_consensus.cc:359] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.629429 28734 raft_consensus.cc:385] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.629532 28734 raft_consensus.cc:740] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:26.629788 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef4a998b01e2437a8edcf95a89e5a314 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.629832 28734 consensus_queue.cc:260] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.630340 28734 ts_tablet_manager.cc:1434] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:26.630669 28734 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.630973 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_864c3add59ac47ec8bbbcb4f4796beee" } modify_external_catalogs: true
I20260320 15:27:26.631196 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_864c3add59ac47ec8bbbcb4f4796beee" } modify_external_catalogs: true
W20260320 15:27:26.632714 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aafd1e123cd043e7828140d2443e047f consensus state has no leader
I20260320 15:27:26.632686 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3036240dd0df4f8785bac1b43a476121"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.633086 28734 tablet_bootstrap.cc:654] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:26.633289 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3036240dd0df4f8785bac1b43a476121 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.637521 28046 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:26.640720 28735 ts_tablet_manager.cc:1916] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.642648 28734 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.642825 28734 ts_tablet_manager.cc:1403] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20260320 15:27:26.643944 28734 raft_consensus.cc:359] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.644181 28734 raft_consensus.cc:385] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.644305 28734 raft_consensus.cc:740] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.644616 28734 consensus_queue.cc:260] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.645257 28734 ts_tablet_manager.cc:1434] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:26.645650 28734 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.650553 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:26.651136 28049 catalog_manager.cc:6741] ReplaceTablet: tablet fa5618796f1a4ff3ab4c1209dad0aede deleted and replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57
I20260320 15:27:26.651608 28734 tablet_bootstrap.cc:654] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.654448 28052 master_service.cc:954] ReplaceTablet: received request to replace tablet da860b779a174adbb4fd6f7f7520348c from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.584802 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.639987 29185 raft_consensus.cc:493] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.655723 29185 raft_consensus.cc:515] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.656524 29185 leader_election.cc:290] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.654434 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.657140 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:26.657454 28447 raft_consensus.cc:2468] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:26.658075 28524 leader_election.cc:304] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:26.658226 28734 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.656864 28584 raft_consensus.cc:2468] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:26.658411 28734 ts_tablet_manager.cc:1403] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260320 15:27:26.658910 28382 leader_election.cc:304] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:26.659293 28734 raft_consensus.cc:359] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.659572 28734 raft_consensus.cc:385] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.659624 29185 raft_consensus.cc:2804] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.659667 28734 raft_consensus.cc:740] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.659772 29185 raft_consensus.cc:493] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.659880 29185 raft_consensus.cc:3060] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.659906 28734 consensus_queue.cc:260] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.660197 29199 raft_consensus.cc:2804] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.660313 29199 raft_consensus.cc:493] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.660508 29199 raft_consensus.cc:3060] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.660516 28734 ts_tablet_manager.cc:1434] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:26.663790 28734 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.665282 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e847499477ab4a5ca57d41599f7605dc" } modify_external_catalogs: true
I20260320 15:27:26.665699 28734 tablet_bootstrap.cc:654] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.665701 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e847499477ab4a5ca57d41599f7605dc" } modify_external_catalogs: true
I20260320 15:27:26.666352 28733 raft_consensus.cc:385] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.666553 28733 raft_consensus.cc:740] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.667068 28733 consensus_queue.cc:260] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.668557 28049 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:26.672165 28049 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]). Expected version 1 got 0
I20260320 15:27:26.672716 28733 ts_tablet_manager.cc:1434] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.103s user 0.000s sys 0.003s
I20260320 15:27:26.674053 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6892575e01594ece9a265373580a05bc" } modify_external_catalogs: true
I20260320 15:27:26.674432 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6892575e01594ece9a265373580a05bc" } modify_external_catalogs: true
I20260320 15:27:26.674834 28733 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:26.675423 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_12077503f4ba430396cb7392d0e64fa7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.676710 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12077503f4ba430396cb7392d0e64fa7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.678017 28733 tablet_bootstrap.cc:654] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.678392 28734 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.678575 28734 ts_tablet_manager.cc:1403] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.002s
I20260320 15:27:26.679426 28734 raft_consensus.cc:359] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.681433 28734 raft_consensus.cc:385] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.681548 28734 raft_consensus.cc:740] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.682248 28734 consensus_queue.cc:260] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.683075 28734 ts_tablet_manager.cc:1434] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260320 15:27:26.684705 28734 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.684669 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:26.686229 28053 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet bdd8f5ccbd35418684fd27459c7a6b9f
I20260320 15:27:26.686977 29156 raft_consensus.cc:493] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.687270 29156 raft_consensus.cc:515] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.688547 29156 leader_election.cc:290] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.690605 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:26.691404 28383 leader_election.cc:343] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.691624 28734 tablet_bootstrap.cc:654] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.692731 28047 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:26.694018 28047 master_service.cc:954] ReplaceTablet: received request to replace tablet ea6ff59d2ffe4d579e6db2375eb11414 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.694694 28917 raft_consensus.cc:515] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.697894 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ba4eb20eb0f1403282030727757031f7 consensus state has no leader
I20260320 15:27:26.699326 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.699240 29199 raft_consensus.cc:515] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.700220 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.695487 28917 leader_election.cc:290] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.702751 29199 leader_election.cc:290] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.703373 28729 ts_tablet_manager.cc:1929] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.703596 28729 log.cc:1199] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/da860b779a174adbb4fd6f7f7520348c
I20260320 15:27:26.704126 28729 ts_tablet_manager.cc:1950] T da860b779a174adbb4fd6f7f7520348c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:26.705459 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 264) took 2256 ms. Trace:
I20260320 15:27:26.705706 28729 rpcz_store.cc:276] 0320 15:27:24.448700 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.449871 (+ 1171us) service_pool.cc:225] Handling call
0320 15:27:26.438834 (+1988963us) ts_tablet_manager.cc:1151] Deleting tablet da860b779a174adbb4fd6f7f7520348c
0320 15:27:26.438848 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.607729 (+168881us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.703193 (+ 95464us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.705411 (+ 2218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1986918}
I20260320 15:27:26.706406 28729 tablet_replica.cc:333] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:26.706696 28729 raft_consensus.cc:2243] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:26.707082 28729 pending_rounds.cc:70] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:26.707324 28729 pending_rounds.cc:77] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387743717806080 op_type: NO_OP noop_request { }
I20260320 15:27:26.707523 28729 raft_consensus.cc:2889] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.707738 28729 raft_consensus.cc:2272] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.707679 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.708539 28729 ts_tablet_manager.cc:1916] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.708655 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.712373 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.713073 29199 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.713348 29199 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.715130 29199 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.717665 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.719136 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:26.722895 28729 ts_tablet_manager.cc:1929] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.722873 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.723065 28729 log.cc:1199] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/da957aed322f46acaf5876c567c2ff94
I20260320 15:27:26.723619 28729 ts_tablet_manager.cc:1950] T da957aed322f46acaf5876c567c2ff94 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:26.724531 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 265) took 2275 ms. Trace:
I20260320 15:27:26.724676 28729 rpcz_store.cc:276] 0320 15:27:24.448795 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.450021 (+ 1226us) service_pool.cc:225] Handling call
0320 15:27:26.706329 (+2256308us) ts_tablet_manager.cc:1151] Deleting tablet da957aed322f46acaf5876c567c2ff94
0320 15:27:26.706341 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.717518 (+ 11177us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.722720 (+ 5202us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.724486 (+ 1766us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2254384}
I20260320 15:27:26.730608 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 266) took 2272 ms. Trace:
I20260320 15:27:26.730799 28729 rpcz_store.cc:276] 0320 15:27:24.458448 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.458522 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:26.730397 (+2271875us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:26.730411 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.730558 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2271618}
I20260320 15:27:26.731305 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet da860b779a174adbb4fd6f7f7520348c (table default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]) successfully deleted
I20260320 15:27:26.732255 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 259) took 2284 ms. Trace:
I20260320 15:27:26.732411 28729 rpcz_store.cc:276] 0320 15:27:24.447620 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.482804 (+ 35184us) service_pool.cc:225] Handling call
0320 15:27:26.732041 (+2249237us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:26.732053 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.732173 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2248952}
I20260320 15:27:26.732782 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet da957aed322f46acaf5876c567c2ff94 (table default.Table_7da45f21aeef45fbb41fbf00df427cb3 [id=c649f9c097404e668b8d377e4a87826a]) successfully deleted
I20260320 15:27:26.733799 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 268) took 2093 ms. Trace:
I20260320 15:27:26.733938 28729 rpcz_store.cc:276] 0320 15:27:24.640411 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.641173 (+ 762us) service_pool.cc:225] Handling call
0320 15:27:26.733572 (+2092399us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:26.733585 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.733755 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2092160}
W20260320 15:27:26.734577 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a072859f39c4c28ae65b0e9afa79d66 already in progress: creating tablet
I20260320 15:27:26.735383 28729 tablet_replica.cc:333] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:26.735670 28729 raft_consensus.cc:2243] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.735903 28729 raft_consensus.cc:2272] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:26.735764 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7f0242a7968541f480367b45cda5d9cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f0242a7968541f480367b45cda5d9cc already in progress: creating tablet
I20260320 15:27:26.738217 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:26.738549 28303 raft_consensus.cc:3060] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:26.739454 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ea6ff59d2ffe4d579e6db2375eb11414 already in progress: creating tablet
I20260320 15:27:26.739919 28729 ts_tablet_manager.cc:1916] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.743602 28303 raft_consensus.cc:2468] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:26.744609 28383 leader_election.cc:304] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:26.746106 28917 raft_consensus.cc:2804] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:26.746273 28917 raft_consensus.cc:697] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:26.746615 28917 consensus_queue.cc:237] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.750116 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:26.750401 28300 raft_consensus.cc:2468] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:26.751083 28383 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:26.751344 28383 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:26.752096 29202 raft_consensus.cc:2804] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.752194 29202 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.752308 29202 raft_consensus.cc:3060] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:26.757014 28736 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.563s user 0.064s sys 0.114s
I20260320 15:27:26.756932 29202 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.757334 28736 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.564s user 0.064s sys 0.114s
I20260320 15:27:26.759747 29202 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:26.764556 28736 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:26.764767 28736 ts_tablet_manager.cc:1403] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.577s user 0.066s sys 0.114s
I20260320 15:27:26.765619 28736 raft_consensus.cc:359] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.766008 28736 raft_consensus.cc:385] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.766095 28736 raft_consensus.cc:740] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.766474 28736 consensus_queue.cc:260] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.768250 28736 ts_tablet_manager.cc:1434] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260320 15:27:26.768764 28736 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:26.680581 28524 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:26.689308 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.694299 29185 raft_consensus.cc:515] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.770789 28382 leader_election.cc:343] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.695907 28730 ts_tablet_manager.cc:1929] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.696138 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.770941 28382 leader_election.cc:304] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.696285 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.700551 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:26.702556 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:26.715087 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:26.726367 29197 raft_consensus.cc:493] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.771821 29202 raft_consensus.cc:2749] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:26.772258 28382 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.773732 28736 tablet_bootstrap.cc:654] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:26.776984 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 96124.1us
I20260320 15:27:26.732806 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:26.779479 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:26.779986 28587 raft_consensus.cc:2393] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:26.780174 28730 log.cc:1199] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/81a1e0957439458083e3a86ac2f01b1e
I20260320 15:27:26.781483 28730 ts_tablet_manager.cc:1950] T 81a1e0957439458083e3a86ac2f01b1e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:26.782023 28382 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:26.782337 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 258) took 1989 ms. Trace:
I20260320 15:27:26.782390 28382 leader_election.cc:304] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.782456 28730 rpcz_store.cc:276] 0320 15:27:24.793168 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.793655 (+ 487us) service_pool.cc:225] Handling call
0320 15:27:26.238612 (+1444957us) ts_tablet_manager.cc:1151] Deleting tablet 81a1e0957439458083e3a86ac2f01b1e
0320 15:27:26.238624 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.616184 (+377560us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.695723 (+ 79539us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.782298 (+ 86575us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1444625}
I20260320 15:27:26.782879 28730 tablet_replica.cc:333] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:26.783010 29202 raft_consensus.cc:2749] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.783078 28730 raft_consensus.cc:2243] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:26.783326 28730 pending_rounds.cc:70] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:26.783390 28730 pending_rounds.cc:77] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387745187102720 op_type: NO_OP noop_request { }
I20260320 15:27:26.784534 28053 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_314b2abee493499f82fb7741c3a20f4c" } new_table_name: "default.Table_6880d724db9a4baab855b37ecb8be8f4" modify_external_catalogs: true
I20260320 15:27:26.786569 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_3c050d31cc0248c7beb57aafd767d53f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.787177 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c050d31cc0248c7beb57aafd767d53f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.793494 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_74ed9d4773ad47bf919a3d16d763795a" } new_table_name: "default.Table_a89976557ff942d7a9b2412273ac664b" modify_external_catalogs: true
I20260320 15:27:26.797199 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a2607c8356f048f1bb3fbe1f240848c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.797972 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2607c8356f048f1bb3fbe1f240848c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.798097 28052 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet aeba9e691b0a4fb7a479e2b5035a7eca
I20260320 15:27:26.787889 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a0e8e2ce7a454ebcab4e59a4de68dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:26.789078 29215 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:26.798791 28300 raft_consensus.cc:3060] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.801774 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ada1f644b754fbe8b3790b46ec8cdf4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:26.802093 28447 raft_consensus.cc:2393] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:26.802285 28300 raft_consensus.cc:2468] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:26.802739 29185 leader_election.cc:290] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.803027 28735 ts_tablet_manager.cc:1929] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.803032 28730 raft_consensus.cc:2889] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.803496 28730 raft_consensus.cc:2272] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.804356 28730 ts_tablet_manager.cc:1916] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.805644 29149 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
W20260320 15:27:26.805647 28524 consensus_peers.cc:597] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d57566e7907c4783aafce347e46b89e3. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.805779 29149 catalog_manager.cc:1562] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: loading cluster ID for follower catalog manager: success
I20260320 15:27:26.780417 29197 raft_consensus.cc:515] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.806888 29197 leader_election.cc:290] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.807099 28586 raft_consensus.cc:3060] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.809136 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_294d5eabb8df40cd888a0769290746bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.809854 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_294d5eabb8df40cd888a0769290746bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.810401 28586 raft_consensus.cc:2468] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:26.812162 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a5ca3ae6cd61430db4e2c4112c011e95" } new_table_name: "default.Table_f81d07130d8943e5bb8ba81230e2e6b7" modify_external_catalogs: true
I20260320 15:27:26.814781 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_fd26120a419a45efa62bf54b58fd74a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.816658 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:26.819890 28524 consensus_peers.cc:597] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d57566e7907c4783aafce347e46b89e3. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:26.819952 28735 log.cc:1199] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0ae0ad80b6a44b91b18fa1e250f7dfbc
I20260320 15:27:26.820163 28051 master_service.cc:954] ReplaceTablet: received request to replace tablet 20a7164db1e0447fb424d8e46b747f52 from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:26.820410 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd26120a419a45efa62bf54b58fd74a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.820673 28735 ts_tablet_manager.cc:1950] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:26.821730 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 285) took 1852 ms. Trace:
I20260320 15:27:26.821857 28735 rpcz_store.cc:276] 0320 15:27:24.968936 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.969815 (+ 879us) service_pool.cc:225] Handling call
0320 15:27:26.569527 (+1599712us) ts_tablet_manager.cc:1151] Deleting tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc
0320 15:27:26.569544 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.759101 (+189557us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.802921 (+ 43820us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.821648 (+ 18727us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1599401}
I20260320 15:27:26.822494 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 289) took 1838 ms. Trace:
I20260320 15:27:26.822611 28735 rpcz_store.cc:276] 0320 15:27:24.984333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.984404 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:26.822270 (+1837866us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:26.822283 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.822399 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1837602}
I20260320 15:27:26.830982 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.831840 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc (table default.Table_714d170555c642ddaa667c974b68cbb7 [id=592cc526af644da5abbc2e49c7e38956]) successfully deleted
I20260320 15:27:26.838673 28735 tablet_replica.cc:333] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:26.839329 28735 raft_consensus.cc:2243] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.840044 28735 raft_consensus.cc:2272] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.843569 28734 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.843844 28734 ts_tablet_manager.cc:1403] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.159s user 0.008s sys 0.033s
I20260320 15:27:26.844887 28734 raft_consensus.cc:359] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.845629 28734 raft_consensus.cc:385] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.845813 28734 raft_consensus.cc:740] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.846122 28734 consensus_queue.cc:260] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.847007 28734 ts_tablet_manager.cc:1434] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:26.847479 28734 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:26.835175 28053 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1ccfd2d5e82448b99f1022954b98c6cc has no consensus state
I20260320 15:27:26.836869 28048 master_service.cc:954] ReplaceTablet: received request to replace tablet 16e31e11048b494ebe1fd4d89488b0cf from {username='slave'} at 127.0.0.1:34644
W20260320 15:27:26.848533 28041 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ecaa08ba34514c5e99a9e2d4ba9268a9 already in progress: creating tablet
W20260320 15:27:26.839807 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c2d7e4d26994a17bf2d2615d6552412 consensus state has no leader
I20260320 15:27:26.849210 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_1642210236684e85bbcfdf569f9ae5d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.849957 28734 tablet_bootstrap.cc:654] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.850093 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5b5bbe0346774e8cbb24c184595d322e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:26.844420 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d78ab2f7e0d7413ea9049dba9cbd8633 consensus state has no leader
I20260320 15:27:26.851265 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a019d578836346b786ea9bfab466a42d" } new_table_name: "default.Table_fa221247f0074abdbafb6ec471027c6a" modify_external_catalogs: true
I20260320 15:27:26.852456 28735 ts_tablet_manager.cc:1916] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.852603 29213 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:26.853154 29214 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260320 15:27:26.861281 28041 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet b4cad900411e4c9686701aa99206dd23 consensus state has no leader
I20260320 15:27:26.863219 28050 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9] alter complete (version 1)
W20260320 15:27:26.871680 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8d92e17cea044541968356ae993be1e7 has no consensus state
I20260320 15:27:26.857769 28734 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.879741 28734 ts_tablet_manager.cc:1403] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.002s
I20260320 15:27:26.879884 28730 ts_tablet_manager.cc:1929] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.880095 28730 log.cc:1199] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/28c1dcc311a14b33a289ae2b8a485d2b
W20260320 15:27:26.880198 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1642210236684e85bbcfdf569f9ae5d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.880729 28730 ts_tablet_manager.cc:1950] T 28c1dcc311a14b33a289ae2b8a485d2b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:26.880698 28734 raft_consensus.cc:359] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.880921 28734 raft_consensus.cc:385] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.881024 28734 raft_consensus.cc:740] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.881310 28734 consensus_queue.cc:260] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.881909 28734 ts_tablet_manager.cc:1434] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:26.859844 28733 tablet_bootstrap.cc:492] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:26.885809 28733 ts_tablet_manager.cc:1403] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.211s user 0.005s sys 0.000s
I20260320 15:27:26.886940 28733 raft_consensus.cc:359] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.887266 28733 raft_consensus.cc:385] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.887423 28733 raft_consensus.cc:740] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.888321 28733 consensus_queue.cc:260] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.891590 28383 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:26.892200 28917 raft_consensus.cc:2804] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:26.892479 28917 raft_consensus.cc:697] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:26.892874 28917 consensus_queue.cc:237] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.894642 28729 ts_tablet_manager.cc:1929] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:26.905014 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b5bbe0346774e8cbb24c184595d322e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.912426 28729 log.cc:1199] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/545c8fd2d4c845adba6dbf42dfddba3e
I20260320 15:27:26.895485 28734 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:26.916353 29197 raft_consensus.cc:493] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:26.916610 28729 ts_tablet_manager.cc:1950] T 545c8fd2d4c845adba6dbf42dfddba3e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:26.916590 29197 raft_consensus.cc:515] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.917479 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 276) took 2231 ms. Trace:
I20260320 15:27:26.917587 28729 rpcz_store.cc:276] 0320 15:27:24.685907 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.685996 (+ 89us) service_pool.cc:225] Handling call
0320 15:27:26.735299 (+2049303us) ts_tablet_manager.cc:1151] Deleting tablet 545c8fd2d4c845adba6dbf42dfddba3e
0320 15:27:26.735312 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.743680 (+ 8368us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.894474 (+150794us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.917418 (+ 22944us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2035831}
I20260320 15:27:26.917845 29197 leader_election.cc:290] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.918382 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:26.918607 28447 raft_consensus.cc:2468] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:26.919070 28524 leader_election.cc:304] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:26.921356 29197 raft_consensus.cc:2804] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:26.901716 29149 catalog_manager.cc:1584] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: acquiring CA information for follower catalog manager: success
I20260320 15:27:26.921538 29197 raft_consensus.cc:493] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:26.921671 29197 raft_consensus.cc:3060] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.923915 28734 tablet_bootstrap.cc:654] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.925612 29197 raft_consensus.cc:515] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.926549 29197 leader_election.cc:290] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:26.927192 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:26.927443 28447 raft_consensus.cc:3060] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.930722 28447 raft_consensus.cc:2468] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:26.931983 28524 leader_election.cc:304] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:26.932387 29197 raft_consensus.cc:2804] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:26.933097 29197 raft_consensus.cc:697] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:26.933544 29197 consensus_queue.cc:237] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.933708 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 277) took 2158 ms. Trace:
W20260320 15:27:26.934471 28524 leader_election.cc:343] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:26.934646 28524 leader_election.cc:304] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.935130 29185 raft_consensus.cc:2749] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.936342 28729 rpcz_store.cc:276] 0320 15:27:24.774952 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.775326 (+ 374us) service_pool.cc:225] Handling call
0320 15:27:26.933489 (+2158163us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:26.933503 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.933635 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2127683}
I20260320 15:27:26.937292 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:26.937635 28303 raft_consensus.cc:2468] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:26.938858 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca38bc990634e21a2c38dd840488f8c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:26.939101 28303 raft_consensus.cc:3060] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:26.940266 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 286) took 1985 ms. Trace:
I20260320 15:27:26.940387 28729 rpcz_store.cc:276] 0320 15:27:24.954747 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.954795 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:26.940024 (+1985229us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:26.940042 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.940189 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1984238}
I20260320 15:27:26.941730 28734 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:26.941907 28734 ts_tablet_manager.cc:1403] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.000s
I20260320 15:27:26.942828 28303 raft_consensus.cc:2468] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:26.943116 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 545c8fd2d4c845adba6dbf42dfddba3e (table default.Table_dabfce1f58cf47eeb405fae7b61d20c3 [id=9b9186ea93394410ac6e71eecf3b24f3]) successfully deleted
W20260320 15:27:26.943962 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet edb7b53bb7444a39b471218c6039542f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb7b53bb7444a39b471218c6039542f already in progress: creating tablet
I20260320 15:27:26.944722 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_65a8a44c03cd4385a65c2e752446c84c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.944981 28736 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:26.944941 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_b3b4c26f9c844ac9a86b26c019bd713e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.945230 28736 ts_tablet_manager.cc:1403] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.177s user 0.004s sys 0.008s
W20260320 15:27:26.945433 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3b4c26f9c844ac9a86b26c019bd713e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.945691 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_608b73cf3b5c48eda1caf8e569c5b5f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:26.946142 28736 raft_consensus.cc:359] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.946280 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_608b73cf3b5c48eda1caf8e569c5b5f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.946354 28736 raft_consensus.cc:385] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.946456 28736 raft_consensus.cc:740] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.946769 28736 consensus_queue.cc:260] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:26.947425 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65a8a44c03cd4385a65c2e752446c84c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:26.951907 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 259) took 2158 ms. Trace:
I20260320 15:27:26.952076 28730 rpcz_store.cc:276] 0320 15:27:24.793339 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.794027 (+ 688us) service_pool.cc:225] Handling call
0320 15:27:26.782814 (+1988787us) ts_tablet_manager.cc:1151] Deleting tablet 28c1dcc311a14b33a289ae2b8a485d2b
0320 15:27:26.782823 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.823425 (+ 40602us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.879680 (+ 56255us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.951857 (+ 72177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1988625}
I20260320 15:27:26.952452 28730 tablet_replica.cc:333] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:26.952764 28733 ts_tablet_manager.cc:1434] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.067s user 0.002s sys 0.000s
I20260320 15:27:26.953073 28733 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:26.954629 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 28c1dcc311a14b33a289ae2b8a485d2b (table default.Table_e976aa12e3ab47bd8d42318871768eba [id=819cfa24d2ea45d7a65f43b9157b4a9f]) successfully deleted
I20260320 15:27:26.954978 28733 tablet_bootstrap.cc:654] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.955461 28736 ts_tablet_manager.cc:1434] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.001s sys 0.002s
I20260320 15:27:26.956218 28730 raft_consensus.cc:2243] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.956435 28730 pending_rounds.cc:70] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:26.956537 28730 pending_rounds.cc:77] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387743713869824 op_type: NO_OP noop_request { }
I20260320 15:27:26.956674 28730 raft_consensus.cc:2889] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:26.956813 28730 raft_consensus.cc:2272] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.957079 28045 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8ad5a03705764f249c6964341f957801" } modify_external_catalogs: true
I20260320 15:27:26.957300 28045 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_8ad5a03705764f249c6964341f957801" } modify_external_catalogs: true
I20260320 15:27:26.957808 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1" } modify_external_catalogs: true
I20260320 15:27:26.958039 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1" } modify_external_catalogs: true
I20260320 15:27:26.958273 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3036240dd0df4f8785bac1b43a476121" } modify_external_catalogs: true
I20260320 15:27:26.958439 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_3036240dd0df4f8785bac1b43a476121" } modify_external_catalogs: true
I20260320 15:27:26.959020 28736 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:26.960254 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:26.960318 28730 ts_tablet_manager.cc:1916] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.962332 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_9cbdeb9adf8f4c8ab1796cc6279e890b" } new_table_name: "default.Table_675a5a632b564fc9b95cd94b474053a7" modify_external_catalogs: true
I20260320 15:27:26.969614 28053 master_service.cc:954] ReplaceTablet: received request to replace tablet 2ca38bc990634e21a2c38dd840488f8c from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.975198 28052 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 92343ff4a95a479a8b64f2a1b6c949fc
W20260320 15:27:26.976081 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8f70142c3a449439a0a475bddafdd40 consensus state has no leader
I20260320 15:27:26.976994 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f591c45d9d73442ab1f0144ddfc8c48b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:26.977277 28300 raft_consensus.cc:2393] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:26.978228 28524 leader_election.cc:304] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:26.978580 29185 raft_consensus.cc:2749] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:26.979465 28735 ts_tablet_manager.cc:1929] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:26.979661 28735 log.cc:1199] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:26.980087 28735 ts_tablet_manager.cc:1950] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:26.980747 28734 raft_consensus.cc:359] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.980957 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 291) took 1938 ms. Trace:
I20260320 15:27:26.981103 28735 rpcz_store.cc:276] 0320 15:27:25.042825 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.042922 (+ 97us) service_pool.cc:225] Handling call
0320 15:27:26.822871 (+1779949us) ts_tablet_manager.cc:1151] Deleting tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
0320 15:27:26.838539 (+ 15668us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.902273 (+ 63734us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.979286 (+ 77013us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.980919 (+ 1633us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1779086}
I20260320 15:27:26.981501 28735 tablet_replica.cc:333] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:26.981724 28735 raft_consensus.cc:2243] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.981905 28735 raft_consensus.cc:2272] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.982388 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]) successfully deleted
I20260320 15:27:26.983063 28735 ts_tablet_manager.cc:1916] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.991603 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 285) took 2025 ms. Trace:
I20260320 15:27:26.991693 28733 tablet_bootstrap.cc:492] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:26.991700 28046 master_service.cc:954] ReplaceTablet: received request to replace tablet a2775196c8e04dcc80c7e089b57579fd from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:26.991771 28729 rpcz_store.cc:276] 0320 15:27:24.954546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.954762 (+ 216us) service_pool.cc:225] Handling call
0320 15:27:26.972143 (+2017381us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:26.972159 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.979784 (+ 7625us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1906234}
I20260320 15:27:26.991888 28733 ts_tablet_manager.cc:1403] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.000s
I20260320 15:27:26.992195 28729 tablet_replica.cc:333] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:26.992352 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c4b53fb135741efad5d3e59a6d4eb3e already in progress: creating tablet
I20260320 15:27:26.992419 28729 raft_consensus.cc:2243] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:26.992632 28729 raft_consensus.cc:2272] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:26.992758 28733 raft_consensus.cc:359] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.993018 28733 raft_consensus.cc:385] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:26.993127 28733 raft_consensus.cc:740] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:26.993227 28729 ts_tablet_manager.cc:1916] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:26.993429 28733 consensus_queue.cc:260] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:26.994081 28733 ts_tablet_manager.cc:1434] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:26.994616 28733 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:26.996743 28733 tablet_bootstrap.cc:654] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:26.998932 28735 ts_tablet_manager.cc:1929] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:26.999647 28730 ts_tablet_manager.cc:1929] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:27.000838 28736 tablet_bootstrap.cc:654] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.001076 28730 log.cc:1199] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/da860b779a174adbb4fd6f7f7520348c
I20260320 15:27:27.001889 28730 ts_tablet_manager.cc:1950] T da860b779a174adbb4fd6f7f7520348c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.002100 28734 raft_consensus.cc:385] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.002256 28734 raft_consensus.cc:740] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.002586 28734 consensus_queue.cc:260] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.003248 28734 ts_tablet_manager.cc:1434] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.061s user 0.002s sys 0.000s
I20260320 15:27:27.005407 28735 log.cc:1199] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:27.005478 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 260) took 2211 ms. Trace:
I20260320 15:27:27.005697 28730 rpcz_store.cc:276] 0320 15:27:24.793541 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.794226 (+ 685us) service_pool.cc:225] Handling call
0320 15:27:26.952368 (+2158142us) ts_tablet_manager.cc:1151] Deleting tablet da860b779a174adbb4fd6f7f7520348c
0320 15:27:26.952383 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.963599 (+ 11216us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.991899 (+ 28300us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.005400 (+ 13501us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3506,"tablet-delete.queue_time_us":2157985}
I20260320 15:27:27.006075 28735 ts_tablet_manager.cc:1950] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:27.007033 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 294) took 1928 ms. Trace:
I20260320 15:27:27.007157 28735 rpcz_store.cc:276] 0320 15:27:25.078912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.079048 (+ 136us) service_pool.cc:225] Handling call
0320 15:27:26.981421 (+1902373us) ts_tablet_manager.cc:1151] Deleting tablet b0e8f37a490a4fdfbc2a3dc56f6da297
0320 15:27:26.981431 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:26.992868 (+ 11437us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:26.998786 (+ 5918us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.006969 (+ 8183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1902035}
I20260320 15:27:27.007793 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet da860b779a174adbb4fd6f7f7520348c (table default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]) successfully deleted
I20260320 15:27:27.008297 28735 tablet_replica.cc:333] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:27.008319 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
I20260320 15:27:27.008522 28735 raft_consensus.cc:2243] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.008746 28735 raft_consensus.cc:2272] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.008935 28730 tablet_replica.cc:333] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.009215 28730 raft_consensus.cc:2243] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:27.010340 28730 raft_consensus.cc:2272] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.010816 28734 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:27.014716 28734 tablet_bootstrap.cc:654] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.016566 28735 ts_tablet_manager.cc:1916] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.018677 29149 catalog_manager.cc:1612] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260320 15:27:27.021133 28730 ts_tablet_manager.cc:1916] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.031667 28054 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2ca38bc990634e21a2c38dd840488f8c (table default.Table_6892575e01594ece9a265373580a05bc [id=96f573af716c4ff5bd62ec012e593f9e]). Expected version 2 got 0
I20260320 15:27:27.036574 28917 raft_consensus.cc:493] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.036761 28917 raft_consensus.cc:515] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.037495 28917 leader_election.cc:290] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.037839 28052 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e4276cc8c36a438b8f87af916b9fe8e3" } modify_external_catalogs: true
I20260320 15:27:27.038035 28052 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e4276cc8c36a438b8f87af916b9fe8e3" } modify_external_catalogs: true
W20260320 15:27:27.042392 28383 leader_election.cc:343] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b395977eb0444bcf8563cd8774fab40f
W20260320 15:27:27.043973 28382 leader_election.cc:343] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:27.044184 28382 leader_election.cc:304] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:27.044998 29202 raft_consensus.cc:2749] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:27.050979 28736 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:27.051327 28736 ts_tablet_manager.cc:1403] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.092s user 0.004s sys 0.000s
I20260320 15:27:27.052214 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 408f5b4d67514838a17c180ae2cd71df (DEFAULT_TABLE table=default.Table_dfd551c35b134e27a2424da18f1a5bd3 [id=b2921bb259f24e1682ec25e2f083ff5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.052384 28736 raft_consensus.cc:359] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.052597 28736 raft_consensus.cc:385] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.052673 28736 raft_consensus.cc:740] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.052754 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408f5b4d67514838a17c180ae2cd71df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.052914 28736 consensus_queue.cc:260] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.053392 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 408f5b4d67514838a17c180ae2cd71df (DEFAULT_TABLE table=default.Table_dfd551c35b134e27a2424da18f1a5bd3 [id=b2921bb259f24e1682ec25e2f083ff5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.053807 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408f5b4d67514838a17c180ae2cd71df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.055909 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 56fa691a26114657ad768545a30dfd3c (DEFAULT_TABLE table=default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.056556 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fa691a26114657ad768545a30dfd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.056630 28730 ts_tablet_manager.cc:1929] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:27.056778 28730 log.cc:1199] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b0e8f37a490a4fdfbc2a3dc56f6da297
I20260320 15:27:27.056782 28736 ts_tablet_manager.cc:1434] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:27.056874 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 408f5b4d67514838a17c180ae2cd71df (DEFAULT_TABLE table=default.Table_dfd551c35b134e27a2424da18f1a5bd3 [id=b2921bb259f24e1682ec25e2f083ff5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.057165 28736 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:27.057240 28730 ts_tablet_manager.cc:1950] T b0e8f37a490a4fdfbc2a3dc56f6da297 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.057934 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408f5b4d67514838a17c180ae2cd71df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.058091 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 257) took 2265 ms. Trace:
I20260320 15:27:27.058194 28730 rpcz_store.cc:276] 0320 15:27:24.793021 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.794500 (+ 1479us) service_pool.cc:225] Handling call
0320 15:27:27.008741 (+2214241us) ts_tablet_manager.cc:1151] Deleting tablet b0e8f37a490a4fdfbc2a3dc56f6da297
0320 15:27:27.008868 (+ 127us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.043644 (+ 34776us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.056460 (+ 12816us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.058028 (+ 1568us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2214045}
I20260320 15:27:27.060046 28917 raft_consensus.cc:493] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.060254 28917 raft_consensus.cc:515] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.060962 28917 leader_election.cc:290] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.062400 28736 tablet_bootstrap.cc:654] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.063740 28730 tablet_replica.cc:333] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.064038 28730 raft_consensus.cc:2243] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.064320 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.064494 28734 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:27.064698 28734 ts_tablet_manager.cc:1403] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.004s
I20260320 15:27:27.064841 28730 raft_consensus.cc:2272] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:27.065104 28383 leader_election.cc:343] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.065677 28734 raft_consensus.cc:359] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.065899 28734 raft_consensus.cc:385] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.066030 28734 raft_consensus.cc:740] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.066354 28734 consensus_queue.cc:260] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.066780 29202 raft_consensus.cc:493] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.066952 29202 raft_consensus.cc:515] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.067706 29202 leader_election.cc:290] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.068687 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.068953 28300 raft_consensus.cc:2468] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:27.069429 28383 leader_election.cc:304] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:27.070299 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b0e8f37a490a4fdfbc2a3dc56f6da297 (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
I20260320 15:27:27.070472 29202 raft_consensus.cc:2804] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:27.070604 29202 raft_consensus.cc:493] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:27.070701 29202 raft_consensus.cc:3060] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.071245 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:27.071511 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:27.071709 28051 catalog_manager.cc:6741] ReplaceTablet: tablet 20a7164db1e0447fb424d8e46b747f52 deleted and replaced by tablet 57cbefe3a19044f78b64db951df5f7c1
I20260320 15:27:27.071965 28048 catalog_manager.cc:6741] ReplaceTablet: tablet 16e31e11048b494ebe1fd4d89488b0cf deleted and replaced by tablet 547d82cee19b4959b3a779c8b8088e83
I20260320 15:27:27.073328 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.073727 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 2ca38bc990634e21a2c38dd840488f8c
I20260320 15:27:27.073982 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.074512 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.075651 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.074949 29202 raft_consensus.cc:515] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.076977 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ca38bc990634e21a2c38dd840488f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.077230 29202 leader_election.cc:290] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.077557 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ca38bc990634e21a2c38dd840488f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.079629 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:27.079890 28300 raft_consensus.cc:3060] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.081911 28729 ts_tablet_manager.cc:1929] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.082078 28729 log.cc:1199] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0ae0ad80b6a44b91b18fa1e250f7dfbc
I20260320 15:27:27.082515 28729 ts_tablet_manager.cc:1950] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:27.083233 28300 raft_consensus.cc:2468] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:27.083521 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 289) took 2014 ms. Trace:
I20260320 15:27:27.083573 28730 ts_tablet_manager.cc:1916] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.085121 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.085131 28729 rpcz_store.cc:276] 0320 15:27:25.069086 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.069169 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:26.992075 (+1922906us) ts_tablet_manager.cc:1151] Deleting tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc
0320 15:27:26.992087 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.043038 (+ 50951us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.047440 (+ 4402us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.083471 (+ 36031us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1913930}
I20260320 15:27:27.085323 29156 raft_consensus.cc:493] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.085512 29156 raft_consensus.cc:515] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.085650 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:27.085829 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 288) took 2017 ms. Trace:
I20260320 15:27:27.085955 28729 rpcz_store.cc:276] 0320 15:27:25.067940 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.071439 (+ 3499us) service_pool.cc:225] Handling call
0320 15:27:27.085652 (+2014213us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:27.085666 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.085784 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1997988}
I20260320 15:27:27.086254 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc (table default.Table_714d170555c642ddaa667c974b68cbb7 [id=592cc526af644da5abbc2e49c7e38956]) successfully deleted
I20260320 15:27:27.086421 29156 leader_election.cc:290] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.086340 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_0f2d272a74ac4a8ab851d662826ba9e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.086548 28729 tablet_replica.cc:333] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:27.086769 28729 raft_consensus.cc:2243] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:27.087090 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ed29146bff74e86a366d3c6329176f4 already in progress: creating tablet
I20260320 15:27:27.087574 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.088886 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_608b73cf3b5c48eda1caf8e569c5b5f5" } new_table_name: "default.Table_952147a3f089453295332fa44b0bf633" modify_external_catalogs: true
I20260320 15:27:27.090759 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_bc3aedc79a434d7b95351602c46df82a" } new_table_name: "default.Table_afb7c7729a7c4fff902d08c295e69c39" modify_external_catalogs: true
W20260320 15:27:27.087109 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f2d272a74ac4a8ab851d662826ba9e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.067044 28734 ts_tablet_manager.cc:1434] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:27.094280 28734 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:27.095297 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_fbb25a9824fc45c7b85abec85ccb75b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.095991 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbb25a9824fc45c7b85abec85ccb75b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.098439 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.099920 28303 raft_consensus.cc:2468] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:27.100880 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:27.101492 28730 ts_tablet_manager.cc:1929] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.101656 28730 log.cc:1199] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/f640a6ae307140d2aee3992333e96d84
I20260320 15:27:27.102072 28730 ts_tablet_manager.cc:1950] T f640a6ae307140d2aee3992333e96d84 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.102180 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_26547ac0e535447fbdbc2aa66519b8fe" } new_table_name: "default.Table_cbe58272fb864ea085279b8bbf91db9a" modify_external_catalogs: true
I20260320 15:27:27.103025 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 261) took 2302 ms. Trace:
I20260320 15:27:27.103155 28730 rpcz_store.cc:276] 0320 15:27:24.800463 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.800661 (+ 198us) service_pool.cc:225] Handling call
0320 15:27:27.063639 (+2262978us) ts_tablet_manager.cc:1151] Deleting tablet f640a6ae307140d2aee3992333e96d84
0320 15:27:27.063652 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.092139 (+ 28487us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.101240 (+ 9101us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.102970 (+ 1730us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2262723}
I20260320 15:27:27.103892 28046 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:27.104368 28046 catalog_manager.cc:6741] ReplaceTablet: tablet a2775196c8e04dcc80c7e089b57579fd deleted and replaced by tablet 8eb3d5e19cda48fda124b6238902c03a
I20260320 15:27:27.104701 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d643a0220e4847efb0a5e33127808a91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.105798 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d643a0220e4847efb0a5e33127808a91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.108055 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.109843 28736 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:27.110368 28736 ts_tablet_manager.cc:1403] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.001s
I20260320 15:27:27.110572 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet a4bfb87d5d5741adb0ad2f2a88f4867f from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:27.112128 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.112416 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ca38bc990634e21a2c38dd840488f8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.112500 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.112739 28736 raft_consensus.cc:359] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.113240 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet f640a6ae307140d2aee3992333e96d84 (table default.Table_fdd90d848d4846dcb37f6b50bce8156b [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
I20260320 15:27:27.113934 28733 tablet_bootstrap.cc:492] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:27.114121 28733 ts_tablet_manager.cc:1403] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.120s user 0.007s sys 0.006s
I20260320 15:27:27.114189 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.114497 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.114802 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 557d8be0e4bf42239b2bb4585e7d0f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.115067 28733 raft_consensus.cc:359] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.115278 28733 raft_consensus.cc:385] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.115383 28733 raft_consensus.cc:740] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.113271 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.114367 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.115649 28733 consensus_queue.cc:260] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.115949 28736 raft_consensus.cc:385] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.116547 28733 ts_tablet_manager.cc:1434] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:27.116671 28730 tablet_replica.cc:333] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.116817 28736 raft_consensus.cc:740] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.116886 28730 raft_consensus.cc:2243] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 LEADER]: Raft consensus shutting down.
I20260320 15:27:27.117242 28730 pending_rounds.cc:70] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:27.117344 28730 pending_rounds.cc:77] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7266387744790601728 op_type: NO_OP noop_request { }
I20260320 15:27:27.117486 28730 raft_consensus.cc:2889] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:27.117815 28736 consensus_queue.cc:260] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.121258 28730 raft_consensus.cc:2272] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.122092 28730 ts_tablet_manager.cc:1916] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.122538 28736 ts_tablet_manager.cc:1434] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260320 15:27:27.122949 28736 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:27.123844 28048 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7b794017a8044d3a82313842a3da9995"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.125392 28048 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b794017a8044d3a82313842a3da9995 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.137830 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 56fa691a26114657ad768545a30dfd3c (DEFAULT_TABLE table=default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.138350 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fa691a26114657ad768545a30dfd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.140847 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 56fa691a26114657ad768545a30dfd3c (DEFAULT_TABLE table=default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.141829 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 6292904782fd4b619483135e58b66684 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.144851 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 (DEFAULT_TABLE table=default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.146291 28736 tablet_bootstrap.cc:654] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.147513 28730 ts_tablet_manager.cc:1929] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260320 15:27:27.147826 28730 log.cc:1199] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/203c619513a146f49a4825962f169f45
I20260320 15:27:27.148450 28730 ts_tablet_manager.cc:1950] T 203c619513a146f49a4825962f169f45 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.148594 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 6d9b0f1936f54c699211c5ee9709f9ed
I20260320 15:27:27.151221 28052 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 41da3cb06b1147fbae2695bc5c9ffaa5
W20260320 15:27:27.152462 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ee4a0d559ce4dfea878bd3f1c03c09a consensus state has no leader
W20260320 15:27:27.152930 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 008811c90aeb4ca4a4ad5c0190a60dcc has no consensus state
I20260320 15:27:27.153592 28735 ts_tablet_manager.cc:1929] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.154219 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.154663 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.157418 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb680bb0863485aaad85a749221e81b (DEFAULT_TABLE table=default.Table_818e1205d4f34148aebe2a4206ca41a5 [id=0998e8ce9720447caf0d5e7fbf145fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.163635 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc (DEFAULT_TABLE table=default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.158056 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.164220 28735 log.cc:1199] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/da860b779a174adbb4fd6f7f7520348c
I20260320 15:27:27.158138 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 6292904782fd4b619483135e58b66684 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.160178 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.161998 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.171965 28047 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c66e0ce947a146eeafcac51f268deaa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.172911 28047 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c66e0ce947a146eeafcac51f268deaa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.173575 28054 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6880d724db9a4baab855b37ecb8be8f4" } new_table_name: "default.Table_3588ac586adb42938b994103229f8dfb" modify_external_catalogs: true
I20260320 15:27:27.176234 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_a5cb91f5ca204475836edc1cb0ee1035"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.176824 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5cb91f5ca204475836edc1cb0ee1035 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.177273 28052 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_4a66f1fda9e9438aad057009144db449"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.177802 28052 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a66f1fda9e9438aad057009144db449 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.178346 28383 leader_election.cc:304] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:27.178969 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fa691a26114657ad768545a30dfd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.179062 28045 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:27.180734 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_da3596fe5dd74584b73521c59103ad57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.181344 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da3596fe5dd74584b73521c59103ad57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.189282 28729 raft_consensus.cc:2272] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:27.190649 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 68762.6us
I20260320 15:27:27.191090 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:27.190555 28382 leader_election.cc:343] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.192297 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.193186 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_5a50b92df46f43e8ac3618740428b8e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.193830 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a50b92df46f43e8ac3618740428b8e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.196475 28383 leader_election.cc:304] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:27.199112 28046 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_675a5a632b564fc9b95cd94b474053a7" } new_table_name: "default.Table_4df4defc37874a3aa612d777259d9d11" modify_external_catalogs: true
I20260320 15:27:27.199796 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:27.200264 28050 catalog_manager.cc:6741] ReplaceTablet: tablet a4bfb87d5d5741adb0ad2f2a88f4867f deleted and replaced by tablet 514e7b02a28142c2b371644089044a9d
I20260320 15:27:27.200277 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 6292904782fd4b619483135e58b66684 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.200497 28917 raft_consensus.cc:2804] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:27.200591 28917 raft_consensus.cc:493] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:27.200577 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c783052b0eb045ea95d77f3722e1ca8b" } modify_external_catalogs: true
I20260320 15:27:27.200703 28917 raft_consensus.cc:3060] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.200752 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_c783052b0eb045ea95d77f3722e1ca8b" } modify_external_catalogs: true
W20260320 15:27:27.200860 28382 leader_election.cc:343] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:27.201598 28382 leader_election.cc:343] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.201747 28382 leader_election.cc:304] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:27.201807 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa54115d4df74bec85970057b5c3dd51 consensus state has no leader
W20260320 15:27:27.202299 28382 leader_election.cc:343] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.202229 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.202636 29156 raft_consensus.cc:2804] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
W20260320 15:27:27.202834 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.202941 29199 raft_consensus.cc:2749] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.205057 28729 ts_tablet_manager.cc:1916] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.207144 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.208993 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:51060
W20260320 15:27:27.211621 28054 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1ccfd2d5e82448b99f1022954b98c6cc consensus state has no leader
I20260320 15:27:27.209352 28418 tablet_service.cc:1511] Processing CreateTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 (DEFAULT_TABLE table=default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.220917 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 3e7b964395f24e45bc2715be3fac6dbe
W20260320 15:27:27.221335 28046 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8f70142c3a449439a0a475bddafdd40 consensus state has no leader
I20260320 15:27:27.227895 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6292904782fd4b619483135e58b66684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.228056 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.230077 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 262) took 2429 ms. Trace:
I20260320 15:27:27.230239 28730 rpcz_store.cc:276] 0320 15:27:24.800603 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:24.800949 (+ 346us) service_pool.cc:225] Handling call
0320 15:27:27.111818 (+2310869us) ts_tablet_manager.cc:1151] Deleting tablet 203c619513a146f49a4825962f169f45
0320 15:27:27.111831 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.130149 (+ 18318us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.133580 (+ 3431us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.230028 (+ 96448us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2310562}
I20260320 15:27:27.230780 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 270) took 2206 ms. Trace:
I20260320 15:27:27.230898 28730 rpcz_store.cc:276] 0320 15:27:25.024376 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.024495 (+ 119us) service_pool.cc:225] Handling call
0320 15:27:27.230528 (+2206033us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:27.230541 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.230666 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2203219}
I20260320 15:27:27.231269 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 272) took 2198 ms. Trace:
I20260320 15:27:27.231387 28730 rpcz_store.cc:276] 0320 15:27:25.033207 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.033373 (+ 166us) service_pool.cc:225] Handling call
0320 15:27:27.231127 (+2197754us) ts_tablet_manager.cc:1151] Deleting tablet 093ae84740d5468188b4a56fa16ded9c
0320 15:27:27.231136 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.231238 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2197425}
I20260320 15:27:27.231789 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 273) took 2198 ms. Trace:
I20260320 15:27:27.231920 28730 rpcz_store.cc:276] 0320 15:27:25.033333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.033774 (+ 441us) service_pool.cc:225] Handling call
0320 15:27:27.231636 (+2197862us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:27.231648 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.231750 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2197651}
I20260320 15:27:27.232033 28917 raft_consensus.cc:515] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.232239 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 274) took 2198 ms. Trace:
I20260320 15:27:27.232342 28730 rpcz_store.cc:276] 0320 15:27:25.034218 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.034276 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:27.232111 (+2197835us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:27.232118 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.232196 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2130841}
I20260320 15:27:27.232647 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 277) took 2165 ms. Trace:
I20260320 15:27:27.232761 28730 rpcz_store.cc:276] 0320 15:27:25.067165 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.067246 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:27.232529 (+2165283us) ts_tablet_manager.cc:1151] Deleting tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
0320 15:27:27.232536 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.232616 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2102620}
I20260320 15:27:27.233057 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 271) took 2200 ms. Trace:
I20260320 15:27:27.233152 28730 rpcz_store.cc:276] 0320 15:27:25.033010 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.034516 (+ 1506us) service_pool.cc:225] Handling call
0320 15:27:27.232948 (+2198432us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:27.232956 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.233027 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2100322}
I20260320 15:27:27.233422 28730 tablet_replica.cc:333] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.233439 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.233642 28730 raft_consensus.cc:2243] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:27.233968 28730 pending_rounds.cc:70] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:27.234059 28730 pending_rounds.cc:77] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387745391046656 op_type: NO_OP noop_request { }
I20260320 15:27:27.234213 28730 raft_consensus.cc:2889] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:27.234521 28730 raft_consensus.cc:2272] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.236044 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6292904782fd4b619483135e58b66684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.236152 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:27.236295 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ccbf4f32636463f8e1e005a629b92cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.239735 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb680bb0863485aaad85a749221e81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.235472 28730 ts_tablet_manager.cc:1916] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.243662 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c453eaf49c4171b0f93d3b26690d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.251720 28917 leader_election.cc:290] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.251802 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc (DEFAULT_TABLE table=default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.251968 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb680bb0863485aaad85a749221e81b (DEFAULT_TABLE table=default.Table_818e1205d4f34148aebe2a4206ca41a5 [id=0998e8ce9720447caf0d5e7fbf145fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.252254 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ccbf4f32636463f8e1e005a629b92cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.253204 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 203c619513a146f49a4825962f169f45 (table default.Table_638e25af5c024f1e93cf9aff051f145f [id=91b1863ad7ad4017a7d937f128f48220]) successfully deleted
W20260320 15:27:27.253511 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7c4b53fb135741efad5d3e59a6d4eb3e already in progress: creating tablet
W20260320 15:27:27.253715 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 093ae84740d5468188b4a56fa16ded9c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 093ae84740d5468188b4a56fa16ded9c already in progress: creating tablet
W20260320 15:27:27.253897 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ed29146bff74e86a366d3c6329176f4 already in progress: creating tablet
W20260320 15:27:27.254056 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ecaa08ba34514c5e99a9e2d4ba9268a9 already in progress: creating tablet
W20260320 15:27:27.254798 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 45a0e8e2ce7a454ebcab4e59a4de68dd already in progress: creating tablet
I20260320 15:27:27.243706 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c453eaf49c4171b0f93d3b26690d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.251746 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:27.256114 28382 leader_election.cc:343] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.257586 28734 tablet_bootstrap.cc:654] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.246892 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.256192 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb680bb0863485aaad85a749221e81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.268354 28729 ts_tablet_manager.cc:1929] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.268570 28729 log.cc:1199] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:27.279562 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6673a2f894aa425bb079ad34bf6940b9" } modify_external_catalogs: true
I20260320 15:27:27.279819 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6673a2f894aa425bb079ad34bf6940b9" } modify_external_catalogs: true
I20260320 15:27:27.292657 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 56fa691a26114657ad768545a30dfd3c
I20260320 15:27:27.294561 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.297331 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.297880 28730 ts_tablet_manager.cc:1929] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:27.298115 28730 log.cc:1199] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/d57566e7907c4783aafce347e46b89e3
I20260320 15:27:27.298664 28730 ts_tablet_manager.cc:1950] T d57566e7907c4783aafce347e46b89e3 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.299458 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 281) took 2105 ms. Trace:
I20260320 15:27:27.299901 28730 rpcz_store.cc:276] 0320 15:27:25.193979 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.194188 (+ 209us) service_pool.cc:225] Handling call
0320 15:27:27.233356 (+2039168us) ts_tablet_manager.cc:1151] Deleting tablet d57566e7907c4783aafce347e46b89e3
0320 15:27:27.233366 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.263167 (+ 29801us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.297686 (+ 34519us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.299409 (+ 1723us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2002522}
I20260320 15:27:27.300779 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 286) took 1878 ms. Trace:
I20260320 15:27:27.300972 28730 rpcz_store.cc:276] 0320 15:27:25.422389 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.422491 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:27.300566 (+1878075us) ts_tablet_manager.cc:1151] Deleting tablet b8f3ee3c611d41b483bd35f868042d05
0320 15:27:27.300578 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.300738 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1877723}
I20260320 15:27:27.302405 29188 raft_consensus.cc:493] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.303856 28051 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:27.304411 28051 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:27.305622 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet d57566e7907c4783aafce347e46b89e3 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]) successfully deleted
I20260320 15:27:27.305919 29188 raft_consensus.cc:515] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:27.306412 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b8f3ee3c611d41b483bd35f868042d05 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8f3ee3c611d41b483bd35f868042d05 already in progress: creating tablet
I20260320 15:27:27.306772 29188 leader_election.cc:290] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.307775 28730 tablet_replica.cc:333] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.309115 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.309895 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.310148 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "993e2fb1561f4317be85e9b5e5816edc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:27.310859 28734 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:27.308859 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "993e2fb1561f4317be85e9b5e5816edc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.311048 28734 ts_tablet_manager.cc:1403] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.217s user 0.002s sys 0.002s
I20260320 15:27:27.312155 28730 raft_consensus.cc:2243] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.312282 28734 raft_consensus.cc:359] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.312623 28734 raft_consensus.cc:385] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.312727 28734 raft_consensus.cc:740] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.312983 28734 consensus_queue.cc:260] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.314417 28733 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:27.315883 28049 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:27.316104 28729 ts_tablet_manager.cc:1950] T 093ae84740d5468188b4a56fa16ded9c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:27.316577 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 287) took 2253 ms. Trace:
I20260320 15:27:27.316679 28729 rpcz_store.cc:276] 0320 15:27:25.063744 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.063842 (+ 98us) service_pool.cc:225] Handling call
0320 15:27:27.086472 (+2022630us) ts_tablet_manager.cc:1151] Deleting tablet 093ae84740d5468188b4a56fa16ded9c
0320 15:27:27.086482 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.239300 (+152818us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.268179 (+ 28879us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.316515 (+ 48336us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9093,"tablet-delete.queue_time_us":1998238}
I20260320 15:27:27.317185 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 093ae84740d5468188b4a56fa16ded9c (table default.Table_20ceb59162bc456f86478ad67ada6956 [id=e79e8aae444f489eb30ffa5b6237e966]) successfully deleted
I20260320 15:27:27.318365 28730 raft_consensus.cc:2272] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.319864 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.321044 28045 master_service.cc:954] ReplaceTablet: received request to replace tablet 796c907919df47bf9369e45a1248f96b from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:27.323122 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 290) took 2228 ms. Trace:
I20260320 15:27:27.323267 28729 rpcz_store.cc:276] 0320 15:27:25.093917 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.094006 (+ 89us) service_pool.cc:225] Handling call
0320 15:27:27.322453 (+2228447us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:27.322476 (+ 23us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.322620 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2228135}
I20260320 15:27:27.323486 28730 ts_tablet_manager.cc:1916] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.326267 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f6315615237b426ca7b2aea87662ecc9" } modify_external_catalogs: true
I20260320 15:27:27.326494 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f6315615237b426ca7b2aea87662ecc9" } modify_external_catalogs: true
I20260320 15:27:27.328094 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7 (DEFAULT_TABLE table=default.Table_5d4b16da449247bda955fe596e458307 [id=3f666b62287a456ca179d4adffd4dbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.328650 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:27.332275 28041 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ecaa08ba34514c5e99a9e2d4ba9268a9 already in progress: creating tablet
I20260320 15:27:27.332859 28730 ts_tablet_manager.cc:1929] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.333135 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet 1c2d7e4d26994a17bf2d2615d6552412 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:27.335548 28730 log.cc:1199] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/f591c45d9d73442ab1f0144ddfc8c48b
I20260320 15:27:27.335822 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.336285 28730 ts_tablet_manager.cc:1950] T f591c45d9d73442ab1f0144ddfc8c48b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.338174 28045 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 796c907919df47bf9369e45a1248f96b
I20260320 15:27:27.338624 28045 catalog_manager.cc:6741] ReplaceTablet: tablet 796c907919df47bf9369e45a1248f96b deleted and replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0
I20260320 15:27:27.338788 28418 tablet_service.cc:1511] Processing CreateTablet for tablet a3bd9c79d2d74e878647f97e66a9bfca (DEFAULT_TABLE table=default.Table_326fcace90ef4e0ba553655d7b3930ae [id=840bfa3d362147309aeded0035f1b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.338913 29156 raft_consensus.cc:697] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:27.339275 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bd9c79d2d74e878647f97e66a9bfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.339236 29156 consensus_queue.cc:237] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.343925 28733 tablet_bootstrap.cc:654] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.346077 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 4becc525b59d421caf79d8cd42052118
I20260320 15:27:27.346522 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:27.347018 28050 catalog_manager.cc:6741] ReplaceTablet: tablet 1c2d7e4d26994a17bf2d2615d6552412 deleted and replaced by tablet 4bd1799035a3431092fbdfad3c42d22b
I20260320 15:27:27.348093 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 288) took 1897 ms. Trace:
I20260320 15:27:27.348433 28730 rpcz_store.cc:276] 0320 15:27:25.444953 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.445005 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:27.307688 (+1862683us) ts_tablet_manager.cc:1151] Deleting tablet f591c45d9d73442ab1f0144ddfc8c48b
0320 15:27:27.307703 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.329524 (+ 21821us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.332720 (+ 3196us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.342412 (+ 9692us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1862355}
I20260320 15:27:27.348313 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7 (DEFAULT_TABLE table=default.Table_5d4b16da449247bda955fe596e458307 [id=3f666b62287a456ca179d4adffd4dbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.349210 29197 raft_consensus.cc:493] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.349422 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.349390 29197 raft_consensus.cc:515] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.350137 29197 leader_election.cc:290] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:27.351395 28050 master_service.cc:954] ReplaceTablet: received request to replace tablet fc009401bb3d440f9d36e080efef1204 from {username='slave'} at 127.0.0.1:34644
I20260320 15:27:27.355034 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 6197f6a1e1cf4007a481be3d7ccfd425 (DEFAULT_TABLE table=default.Table_19d7fded966f4dfcad98437bdbc0e75c [id=fc0da964b571404bbd36fc2257c95c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.355837 28551 tablet_service.cc:1511] Processing CreateTablet for tablet a3bd9c79d2d74e878647f97e66a9bfca (DEFAULT_TABLE table=default.Table_326fcace90ef4e0ba553655d7b3930ae [id=840bfa3d362147309aeded0035f1b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.357945 28734 ts_tablet_manager.cc:1434] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260320 15:27:27.358680 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet fc009401bb3d440f9d36e080efef1204
I20260320 15:27:27.359155 28050 catalog_manager.cc:6741] ReplaceTablet: tablet fc009401bb3d440f9d36e080efef1204 deleted and replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc
I20260320 15:27:27.359858 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6292904782fd4b619483135e58b66684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.360200 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 7bb680bb0863485aaad85a749221e81b (DEFAULT_TABLE table=default.Table_818e1205d4f34148aebe2a4206ca41a5 [id=0998e8ce9720447caf0d5e7fbf145fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.360965 28040 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet f591c45d9d73442ab1f0144ddfc8c48b (table default.Table_764f1d7877d24d50b17c6d27527fdd6d [id=cb3587c86ae2454fadf5647b0e3f92eb]) successfully deleted
I20260320 15:27:27.361186 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 (DEFAULT_TABLE table=default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.361495 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
W20260320 15:27:27.360023 28240 leader_election.cc:343] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:27.362257 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 51464.3us
I20260320 15:27:27.364018 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a9395e2f7cc54d5eac9e394cdb342ad9" } modify_external_catalogs: true
I20260320 15:27:27.364280 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_a9395e2f7cc54d5eac9e394cdb342ad9" } modify_external_catalogs: true
I20260320 15:27:27.364287 28268 tablet_service.cc:1511] Processing CreateTablet for tablet dd315b5d49104d679b03515b65ee30ac (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.364408 28279 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7 (DEFAULT_TABLE table=default.Table_5d4b16da449247bda955fe596e458307 [id=3f666b62287a456ca179d4adffd4dbcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.365101 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 6197f6a1e1cf4007a481be3d7ccfd425 (DEFAULT_TABLE table=default.Table_19d7fded966f4dfcad98437bdbc0e75c [id=fc0da964b571404bbd36fc2257c95c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.365630 28273 tablet_service.cc:1511] Processing CreateTablet for tablet a3bd9c79d2d74e878647f97e66a9bfca (DEFAULT_TABLE table=default.Table_326fcace90ef4e0ba553655d7b3930ae [id=840bfa3d362147309aeded0035f1b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.365993 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.366348 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 8428019c3b074b17aec493c14cc2b911 (DEFAULT_TABLE table=default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.366747 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.366950 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.367249 28269 tablet_service.cc:1511] Processing CreateTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.367803 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e17cea044541968356ae993be1e7 (DEFAULT_TABLE table=default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.368794 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (DEFAULT_TABLE table=default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.369920 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.372219 28051 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_618b12e348124dfab7d293a70b297a41" } new_table_name: "default.Table_36d13972559c46e7861b4c3a0fbbf8b3" modify_external_catalogs: true
I20260320 15:27:27.373080 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:27.373082 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_09f557af64504af69dce683cef4b27fd" } new_table_name: "default.Table_9a40cefec6fa4bb9b70743b12241badd" modify_external_catalogs: true
I20260320 15:27:27.373271 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:27.374102 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c453eaf49c4171b0f93d3b26690d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.374418 28050 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7a91c52621c94498bc3d4d579569893c" } modify_external_catalogs: true
W20260320 15:27:27.374782 28239 leader_election.cc:343] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.374858 28050 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_7a91c52621c94498bc3d4d579569893c" } modify_external_catalogs: true
I20260320 15:27:27.375351 28239 leader_election.cc:304] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:27.375784 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc (DEFAULT_TABLE table=default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.376405 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 5de834725bda4f249882914f1247ab62 (DEFAULT_TABLE table=default.Table_f38d66983ac842a4bb44792143adeeca [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.377097 28300 raft_consensus.cc:3060] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:27.383483 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dd315b5d49104d679b03515b65ee30ac consensus state has no leader
W20260320 15:27:27.384187 28051 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db81df6d3e9d49efb94c4f24de9db429 consensus state has no leader
I20260320 15:27:27.384472 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_186af3c003da48d78f6c983d6f7b63d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.385138 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_186af3c003da48d78f6c983d6f7b63d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.386567 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.389460 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.389671 28917 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.391106 28729 tablet_replica.cc:333] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:27.391319 28269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ccfd2d5e82448b99f1022954b98c6cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.391335 28729 raft_consensus.cc:2243] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:27.391841 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 8428019c3b074b17aec493c14cc2b911 (DEFAULT_TABLE table=default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.392252 28729 pending_rounds.cc:70] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:27.392319 28729 pending_rounds.cc:77] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387750476668928 op_type: NO_OP noop_request { }
I20260320 15:27:27.393359 28050 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:27.395519 28426 tablet_service.cc:1511] Processing CreateTablet for tablet dd315b5d49104d679b03515b65ee30ac (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.396564 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 6197f6a1e1cf4007a481be3d7ccfd425 (DEFAULT_TABLE table=default.Table_19d7fded966f4dfcad98437bdbc0e75c [id=fc0da964b571404bbd36fc2257c95c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.396938 28412 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.397157 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:27.397990 28411 tablet_service.cc:1511] Processing CreateTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.398396 28917 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.399066 28917 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.399339 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_103c6e9e031144a1bc44f351aa73d587"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.399981 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_103c6e9e031144a1bc44f351aa73d587 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.400338 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:27.400471 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 63877us
I20260320 15:27:27.400734 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.400981 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd315b5d49104d679b03515b65ee30ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.401403 28734 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:27.401554 28382 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.401876 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8428019c3b074b17aec493c14cc2b911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.401896 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.402159 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.402390 28303 raft_consensus.cc:2468] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:27.402859 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd315b5d49104d679b03515b65ee30ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.403952 28383 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:27.404449 28917 raft_consensus.cc:2804] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:27.404562 28917 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:27.404670 28917 raft_consensus.cc:3060] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.408671 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ccfd2d5e82448b99f1022954b98c6cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.409825 29230 raft_consensus.cc:493] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.410027 29230 raft_consensus.cc:515] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.410961 29230 leader_election.cc:290] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.412000 28917 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.413000 28917 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.414448 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "557d8be0e4bf42239b2bb4585e7d0f06" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.414812 28297 raft_consensus.cc:2393] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:27.414772 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_e138abbe7dcc46d1babaa051f60268e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.415481 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e138abbe7dcc46d1babaa051f60268e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.416419 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:27.417188 28382 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.418610 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f12e9706fc0f484da670771a01dd860c" } new_table_name: "default.Table_1590be7e3ed84ee2b141222c66efbe1c" modify_external_catalogs: true
I20260320 15:27:27.421588 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e17cea044541968356ae993be1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.422475 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6197f6a1e1cf4007a481be3d7ccfd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.423780 28052 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e6270dc01722414ab060a44ccc4125fa" } new_table_name: "default.Table_5dabb37ba3784a72b87e63c63b6280aa" modify_external_catalogs: true
I20260320 15:27:27.424031 28049 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_f051dee79ff04e4faa3f02332b20d6e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.424667 28049 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f051dee79ff04e4faa3f02332b20d6e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.425231 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_c424c1cd249645e3bed5b49bdf5f75a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.425774 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c424c1cd249645e3bed5b49bdf5f75a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.423354 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_213392d2bde147b2aae83c2b1e8eb8ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.426656 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_213392d2bde147b2aae83c2b1e8eb8ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.427162 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb1733e7a0e4774a5be2e06208bb495 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.429549 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.429768 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 7e13c75a376c47c9a0083173865ce4fa (DEFAULT_TABLE table=default.Table_fb580f5b6e1141e1a236805952cf1214 [id=a497706a56ab423cb61065e803c64897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.429833 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 (DEFAULT_TABLE table=default.Table_e39f3c720ae24bf5978338b793412e12 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.429561 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.430689 28281 tablet_service.cc:1511] Processing CreateTablet for tablet c7e3a6fa7dc74de286177a285dd3d877 (DEFAULT_TABLE table=default.Table_094bdcada4784500a661a56536bb221b [id=5bed4e07a5c04c2f8950f42423d35bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.432250 29188 raft_consensus.cc:2749] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.432422 29043 mvcc.cc:204] Tried to move back new op lower bound from 7266387735101120512 to 7266387716251615232. Current Snapshot: MvccSnapshot[applied={T|T < 7266387735101120512 or (T in {7266387735101120512})}]
I20260320 15:27:27.434607 28050 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_2eff9dcb23c04d50b86d3ad60677435c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.435258 28050 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2eff9dcb23c04d50b86d3ad60677435c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.436182 28729 raft_consensus.cc:2889] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:27.436355 28729 raft_consensus.cc:2272] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.438997 28054 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_57819bbc59d24aa1a4d2a28ac5e50728"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.439236 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_86f6401b78c540eea6cb427c9bd38f6f" } modify_external_catalogs: true
I20260320 15:27:27.439587 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_86f6401b78c540eea6cb427c9bd38f6f" } modify_external_catalogs: true
W20260320 15:27:27.439782 28054 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57819bbc59d24aa1a4d2a28ac5e50728 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.439904 28729 ts_tablet_manager.cc:1916] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.439237 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_80afab5a1d064b83b4e752b4d4c0cf61" } new_table_name: "default.Table_ec3f886f78ed4f3092a8d4e6cf282475" modify_external_catalogs: true
I20260320 15:27:27.442914 28419 tablet_service.cc:1511] Processing CreateTablet for tablet c7e3a6fa7dc74de286177a285dd3d877 (DEFAULT_TABLE table=default.Table_094bdcada4784500a661a56536bb221b [id=5bed4e07a5c04c2f8950f42423d35bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.445199 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.445444 28300 raft_consensus.cc:2468] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:27.445822 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.446317 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8428019c3b074b17aec493c14cc2b911. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:27.447470 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c7a605bdcf14d2ab64ad078efd3d999 consensus state has no leader
W20260320 15:27:27.448041 28052 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f5beaa1bc1e4465be34c8e86f131559 consensus state has no leader
I20260320 15:27:27.447949 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_d4436c4c1a2a4ad1af830b4d62491073"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.448069 28045 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_8aca34fa920c499eb7e2ae000ced9d02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.448647 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4436c4c1a2a4ad1af830b4d62491073 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:27.448722 28045 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8aca34fa920c499eb7e2ae000ced9d02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.450265 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cb44b9cc6445cf8324cf52190d6773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.450565 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb1733e7a0e4774a5be2e06208bb495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.450789 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e3a6fa7dc74de286177a285dd3d877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.450953 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d547c032332d4d109222b4743fa28561" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:27.451215 28303 raft_consensus.cc:3060] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.453064 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e13c75a376c47c9a0083173865ce4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.454947 28303 raft_consensus.cc:2468] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:27.455804 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.456248 28283 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed (DEFAULT_TABLE table=default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.456480 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.456743 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b0f1936f54c699211c5ee9709f9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.457005 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bd9c79d2d74e878647f97e66a9bfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.462800 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6197f6a1e1cf4007a481be3d7ccfd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.466961 28267 tablet_service.cc:1511] Processing CreateTablet for tablet ad4a30476e7146afae1e77dad078136e (DEFAULT_TABLE table=default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.467469 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4a30476e7146afae1e77dad078136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.470714 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ccbf4f32636463f8e1e005a629b92cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.475587 29188 raft_consensus.cc:493] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.476644 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 3ccbf4f32636463f8e1e005a629b92cc
I20260320 15:27:27.480557 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3bd9c79d2d74e878647f97e66a9bfca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.480640 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 (DEFAULT_TABLE table=default.Table_e39f3c720ae24bf5978338b793412e12 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.481098 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:27.483325 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ba738a76a424c18b36985a1977bcee4 consensus state has no leader
I20260320 15:27:27.483645 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.483695 28524 leader_election.cc:304] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:27.484606 28554 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.480777 29188 raft_consensus.cc:515] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.486670 29188 leader_election.cc:290] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.487318 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de834725bda4f249882914f1247ab62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.488125 29188 raft_consensus.cc:493] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.488309 29188 raft_consensus.cc:515] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.489292 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 7e13c75a376c47c9a0083173865ce4fa (DEFAULT_TABLE table=default.Table_fb580f5b6e1141e1a236805952cf1214 [id=a497706a56ab423cb61065e803c64897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.488282 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed (DEFAULT_TABLE table=default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.490438 29188 leader_election.cc:290] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.490876 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e17cea044541968356ae993be1e7 (DEFAULT_TABLE table=default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.491555 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e3a6fa7dc74de286177a285dd3d877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.492913 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (DEFAULT_TABLE table=default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.494273 29197 raft_consensus.cc:3060] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.495738 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6197f6a1e1cf4007a481be3d7ccfd425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.498348 28383 leader_election.cc:304] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
W20260320 15:27:27.498836 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 52704.3us
I20260320 15:27:27.498924 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2355eadcae6483bae7ae475d74dd131" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:27.488168 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:27.496973 28382 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.500311 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_fdd90d848d4846dcb37f6b50bce8156b" } modify_external_catalogs: true
I20260320 15:27:27.500563 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_fdd90d848d4846dcb37f6b50bce8156b" } modify_external_catalogs: true
I20260320 15:27:27.494103 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bb680bb0863485aaad85a749221e81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.503326 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 8d92e17cea044541968356ae993be1e7 (DEFAULT_TABLE table=default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.503969 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.504159 28563 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.504274 28549 tablet_service.cc:1511] Processing CreateTablet for tablet dd315b5d49104d679b03515b65ee30ac (DEFAULT_TABLE table=default.Table_09f557af64504af69dce683cef4b27fd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.505036 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 5de834725bda4f249882914f1247ab62 (DEFAULT_TABLE table=default.Table_f38d66983ac842a4bb44792143adeeca [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.505318 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 8428019c3b074b17aec493c14cc2b911 (DEFAULT_TABLE table=default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.505591 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2355eadcae6483bae7ae475d74dd131" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.507088 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed (DEFAULT_TABLE table=default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.507647 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3ee3c611d41b483bd35f868042d05" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.513406 28565 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.513396 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 7e13c75a376c47c9a0083173865ce4fa (DEFAULT_TABLE table=default.Table_fb580f5b6e1141e1a236805952cf1214 [id=a497706a56ab423cb61065e803c64897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.515308 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 (DEFAULT_TABLE table=default.Table_e39f3c720ae24bf5978338b793412e12 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:27.519788 28240 leader_election.cc:343] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.507663 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 289) took 2038 ms. Trace:
I20260320 15:27:27.530260 28268 tablet_service.cc:1511] Processing CreateTablet for tablet cea3bf14653f4555acfed1577a79ee4f (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.531894 28411 tablet_service.cc:1511] Processing CreateTablet for tablet 5de834725bda4f249882914f1247ab62 (DEFAULT_TABLE table=default.Table_f38d66983ac842a4bb44792143adeeca [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.532992 28423 tablet_service.cc:1511] Processing CreateTablet for tablet cea3bf14653f4555acfed1577a79ee4f (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.533242 29156 raft_consensus.cc:2804] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:27.533439 28414 tablet_service.cc:1511] Processing CreateTablet for tablet ad4a30476e7146afae1e77dad078136e (DEFAULT_TABLE table=default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.533753 29156 raft_consensus.cc:697] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:27.533828 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.534070 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 44327000792444b89265f4a106eb11d0 (DEFAULT_TABLE table=default.Table_3d0dcd71707248d49f09c2356ee00f6c [id=98dd29a311a548d3bc64a6948ac8743c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:27.534407 28383 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.534337 29156 consensus_queue.cc:237] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.534551 28383 leader_election.cc:304] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:27.534683 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b0f1936f54c699211c5ee9709f9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.535084 28383 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:27.535938 29230 raft_consensus.cc:2749] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.536191 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4a30476e7146afae1e77dad078136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.536298 29202 consensus_queue.cc:1048] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:27.537595 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de834725bda4f249882914f1247ab62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.537930 28736 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:27.536201 29230 raft_consensus.cc:2804] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:27.538542 29230 raft_consensus.cc:697] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:27.538829 29230 consensus_queue.cc:237] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.539848 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44327000792444b89265f4a106eb11d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.539856 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb1733e7a0e4774a5be2e06208bb495 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.518656 28563 tablet_service.cc:1511] Processing CreateTablet for tablet c7e3a6fa7dc74de286177a285dd3d877 (DEFAULT_TABLE table=default.Table_094bdcada4784500a661a56536bb221b [id=5bed4e07a5c04c2f8950f42423d35bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.541376 28557 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.541857 28565 tablet_service.cc:1511] Processing CreateTablet for tablet ad4a30476e7146afae1e77dad078136e (DEFAULT_TABLE table=default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.545686 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.557269 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7feb8477ebf841a19977a434f365f537
I20260320 15:27:27.570643 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.569176 28730 rpcz_store.cc:276] 0320 15:27:25.446839 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.447112 (+ 273us) service_pool.cc:225] Handling call
0320 15:27:27.348967 (+1901855us) ts_tablet_manager.cc:1151] Deleting tablet d547c032332d4d109222b4743fa28561
0320 15:27:27.348979 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.485595 (+136616us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1901574}
I20260320 15:27:27.576826 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3ee3c611d41b483bd35f868042d05" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:27.577165 28447 raft_consensus.cc:2468] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:27.580370 28051 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_82d375c404eb4fdca024e9fc85f87e1a" } modify_external_catalogs: true
I20260320 15:27:27.580672 28051 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_82d375c404eb4fdca024e9fc85f87e1a" } modify_external_catalogs: true
I20260320 15:27:27.581074 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (DEFAULT_TABLE table=default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.581336 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e17cea044541968356ae993be1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.581527 28049 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e9195525674d41a89bd4e262baa86a81" } modify_external_catalogs: true
I20260320 15:27:27.582880 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:27.583459 28049 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e9195525674d41a89bd4e262baa86a81" } modify_external_catalogs: true
I20260320 15:27:27.584882 28551 tablet_service.cc:1511] Processing CreateTablet for tablet cea3bf14653f4555acfed1577a79ee4f (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.586591 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea3bf14653f4555acfed1577a79ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.586866 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 7fb1733e7a0e4774a5be2e06208bb495 (DEFAULT_TABLE table=default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.592059 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.593925 28045 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_da3596fe5dd74584b73521c59103ad57" } new_table_name: "default.Table_0c66e258345e4633aea4d4962f922159" modify_external_catalogs: true
I20260320 15:27:27.597986 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41da3cb06b1147fbae2695bc5c9ffaa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.599254 28047 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_e39f3c720ae24bf5978338b793412e12" } new_table_name: "default.Table_add1cf33998542f5afe4fd5216b91bf6" modify_external_catalogs: true
I20260320 15:27:27.601536 28240 leader_election.cc:304] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:27.602373 29188 raft_consensus.cc:2804] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:27.605144 28048 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6cadeb80b9df43149080a924bb38ae8e" } modify_external_catalogs: true
I20260320 15:27:27.605448 28048 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_6cadeb80b9df43149080a924bb38ae8e" } modify_external_catalogs: true
I20260320 15:27:27.616207 28734 tablet_bootstrap.cc:654] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.616751 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92e17cea044541968356ae993be1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.620419 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e13c75a376c47c9a0083173865ce4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.623371 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cb44b9cc6445cf8324cf52190d6773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.623361 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea3bf14653f4555acfed1577a79ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.623420 28917 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.627830 29188 raft_consensus.cc:493] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:27.627830 28735 ts_tablet_manager.cc:1950] T da860b779a174adbb4fd6f7f7520348c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:27.628041 28917 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.629050 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41da3cb06b1147fbae2695bc5c9ffaa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.625123 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.632716 28917 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.638623 28054 catalog_manager.cc:5671] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:27.639158 28054 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
W20260320 15:27:27.641831 28239 leader_election.cc:343] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:27.646368 28040 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet d547c032332d4d109222b4743fa28561 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d547c032332d4d109222b4743fa28561 already in progress: creating tablet
I20260320 15:27:27.650777 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.654311 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ccfd2d5e82448b99f1022954b98c6cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.655164 28730 tablet_replica.cc:333] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.657030 29188 raft_consensus.cc:3060] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.657840 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:51060
I20260320 15:27:27.659544 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 44327000792444b89265f4a106eb11d0 (DEFAULT_TABLE table=default.Table_3d0dcd71707248d49f09c2356ee00f6c [id=98dd29a311a548d3bc64a6948ac8743c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.661476 28730 raft_consensus.cc:2243] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.661679 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.663300 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 44327000792444b89265f4a106eb11d0 (DEFAULT_TABLE table=default.Table_3d0dcd71707248d49f09c2356ee00f6c [id=98dd29a311a548d3bc64a6948ac8743c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.668085 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.670601 28239 leader_election.cc:304] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:27.671232 28239 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.663739 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 297) took 2499 ms. Trace:
I20260320 15:27:27.677510 28730 raft_consensus.cc:2272] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.677575 28735 rpcz_store.cc:276] 0320 15:27:25.157354 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.173343 (+ 15989us) service_pool.cc:225] Handling call
0320 15:27:27.008185 (+1834842us) ts_tablet_manager.cc:1151] Deleting tablet da860b779a174adbb4fd6f7f7520348c
0320 15:27:27.008199 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.020920 (+ 12721us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.153402 (+132482us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.656406 (+503004us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1834487}
W20260320 15:27:27.677743 28382 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.677971 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb1733e7a0e4774a5be2e06208bb495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.678026 28735 tablet_replica.cc:333] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:27.678268 28730 ts_tablet_manager.cc:1916] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.678323 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44327000792444b89265f4a106eb11d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.679632 28736 ts_tablet_manager.cc:1403] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.416s user 0.020s sys 0.010s
I20260320 15:27:27.684000 29237 raft_consensus.cc:493] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.684788 28303 raft_consensus.cc:2468] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:27.684252 29237 raft_consensus.cc:515] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.687722 29237 leader_election.cc:290] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:27.687736 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.685334 28383 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:27.689284 28303 raft_consensus.cc:2393] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:27.692365 29197 raft_consensus.cc:2749] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.691357 28917 raft_consensus.cc:2804] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:27.693631 28917 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:27.696014 28917 raft_consensus.cc:3060] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.698021 29188 raft_consensus.cc:515] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.698496 28735 raft_consensus.cc:2243] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.699795 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3ee3c611d41b483bd35f868042d05" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:27.700306 28729 ts_tablet_manager.cc:1929] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260320 15:27:27.700528 28239 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.700938 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d9b0f1936f54c699211c5ee9709f9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.701476 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06cb44b9cc6445cf8324cf52190d6773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.702355 28917 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.703234 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f6f683638874e90b0d3c8969c702a5c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:27.704015 28524 leader_election.cc:304] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:27.705273 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:27.705521 28303 raft_consensus.cc:3060] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.707739 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41da3cb06b1147fbae2695bc5c9ffaa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.708073 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea3bf14653f4555acfed1577a79ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.708045 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f3ee3c611d41b483bd35f868042d05" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:27.708173 29188 leader_election.cc:290] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.708571 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e3a6fa7dc74de286177a285dd3d877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.709389 29199 consensus_queue.cc:1048] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260320 15:27:27.711273 29237 raft_consensus.cc:3060] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.711860 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44327000792444b89265f4a106eb11d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.714649 29237 raft_consensus.cc:2749] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.714982 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fb1733e7a0e4774a5be2e06208bb495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.719630 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad4a30476e7146afae1e77dad078136e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.724709 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd315b5d49104d679b03515b65ee30ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.732582 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8428019c3b074b17aec493c14cc2b911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.736521 28444 raft_consensus.cc:3060] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:27.748853 28051 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 71ec63fb080545de838803033f204fab
I20260320 15:27:27.752353 29199 raft_consensus.cc:493] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.752818 28735 raft_consensus.cc:2272] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.753810 28736 raft_consensus.cc:359] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.754670 28303 raft_consensus.cc:2468] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:27.762365 28051 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_602a7434dd7d43a798b06bff668c8d0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.762809 28050 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_f38d66983ac842a4bb44792143adeeca" } new_table_name: "default.Table_aff011445f764f9abb08fc4c8d68fc4d" modify_external_catalogs: true
W20260320 15:27:27.763139 28051 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_602a7434dd7d43a798b06bff668c8d0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.763324 28735 ts_tablet_manager.cc:1916] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.763706 28736 raft_consensus.cc:385] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.763811 28736 raft_consensus.cc:740] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.763867 28046 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:27.764081 28736 consensus_queue.cc:260] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:27.764454 28046 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.765640 29188 raft_consensus.cc:493] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.765815 29188 raft_consensus.cc:515] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.766537 29188 leader_election.cc:290] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:27.766880 29236 log.cc:927] Time spent T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260320 15:27:27.769942 28053 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34644:
name: "default.Table_32e1ad85cf09464a87ffd6feb70093cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:27.770749 28053 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32e1ad85cf09464a87ffd6feb70093cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:27.776829 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (DEFAULT_TABLE table=default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.777874 28736 ts_tablet_manager.cc:1434] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.098s user 0.002s sys 0.000s
I20260320 15:27:27.778399 28736 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:27.789880 28736 tablet_bootstrap.cc:654] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:27.796566 29233 raft_consensus.cc:2749] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:27.799161 28917 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.799391 28729 log.cc:1199] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:27.799703 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "093ae84740d5468188b4a56fa16ded9c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.801311 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:27.804522 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de834725bda4f249882914f1247ab62. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:27.804605 28239 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.805092 29199 raft_consensus.cc:515] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.805289 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 85c397eab0ed4df188b0dc2a58077b39 (DEFAULT_TABLE table=default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 [id=667652afa87d46bc878b41081fc53171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.807051 29199 leader_election.cc:290] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.807166 28733 tablet_bootstrap.cc:492] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:27.810262 28383 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:27.810927 28917 raft_consensus.cc:2804] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:27.811350 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:27.812269 28733 ts_tablet_manager.cc:1403] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.493s user 0.006s sys 0.010s
I20260320 15:27:27.813534 28736 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:27.817129 28736 ts_tablet_manager.cc:1403] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.005s
I20260320 15:27:27.813403 28735 ts_tablet_manager.cc:1929] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.827178 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (DEFAULT_TABLE table=default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.827948 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c7a605bdcf14d2ab64ad078efd3d999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.820765 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e13c75a376c47c9a0083173865ce4fa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:27.832020 28383 leader_election.cc:343] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.822283 28733 raft_consensus.cc:359] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.832324 28733 raft_consensus.cc:385] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.832423 28733 raft_consensus.cc:740] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:27.832587 28382 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.832710 28733 consensus_queue.cc:260] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.833825 28733 ts_tablet_manager.cc:1434] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20260320 15:27:27.834250 28733 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:27.834461 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85c397eab0ed4df188b0dc2a58077b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.838459 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.838397 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 85c397eab0ed4df188b0dc2a58077b39 (DEFAULT_TABLE table=default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 [id=667652afa87d46bc878b41081fc53171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.841042 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85c397eab0ed4df188b0dc2a58077b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.845768 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c7a605bdcf14d2ab64ad078efd3d999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.849574 28917 raft_consensus.cc:697] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:27.858966 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet da860b779a174adbb4fd6f7f7520348c (table default.Table_69c8a46cb085490e8a0e7b689e10fa36 [id=e6d1b5562d624822861ce71366259ea6]) successfully deleted
I20260320 15:27:27.859098 29233 raft_consensus.cc:493] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:27.867918 28729 ts_tablet_manager.cc:1950] T 45a0e8e2ce7a454ebcab4e59a4de68dd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:27.877069 28917 consensus_queue.cc:237] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.878388 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (DEFAULT_TABLE table=default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.878906 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c7a605bdcf14d2ab64ad078efd3d999. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.879880 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 85c397eab0ed4df188b0dc2a58077b39 (DEFAULT_TABLE table=default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 [id=667652afa87d46bc878b41081fc53171]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:27.895735 28412 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
W20260320 15:27:27.896133 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 48183.6us
I20260320 15:27:27.899580 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.902030 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85c397eab0ed4df188b0dc2a58077b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:27.904120 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 294) took 2709 ms. Trace:
W20260320 15:27:27.911741 28382 leader_election.cc:343] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.921660 28730 ts_tablet_manager.cc:1929] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:27.921833 28730 log.cc:1199] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0ae0ad80b6a44b91b18fa1e250f7dfbc
I20260320 15:27:27.922266 28730 ts_tablet_manager.cc:1950] T 0ae0ad80b6a44b91b18fa1e250f7dfbc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:27.923123 28733 tablet_bootstrap.cc:654] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:27.929028 28040 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 103939us
I20260320 15:27:27.820783 28736 raft_consensus.cc:359] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.914240 28729 rpcz_store.cc:276] 0320 15:27:25.163187 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.163259 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:27.343476 (+2180217us) ts_tablet_manager.cc:1151] Deleting tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
0320 15:27:27.343487 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.602430 (+258943us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.690030 (+ 87600us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.868320 (+178290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2138294}
I20260320 15:27:27.937625 29237 consensus_queue.cc:1048] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:27.938198 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 290) took 2422 ms. Trace:
I20260320 15:27:27.938342 28730 rpcz_store.cc:276] 0320 15:27:25.515655 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.515755 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:27.574835 (+2059080us) ts_tablet_manager.cc:1151] Deleting tablet 0ae0ad80b6a44b91b18fa1e250f7dfbc
0320 15:27:27.574982 (+ 147us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.742773 (+167791us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.921480 (+178707us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.938148 (+ 16668us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15449,"tablet-delete.queue_time_us":2058053}
I20260320 15:27:27.938868 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 291) took 2385 ms. Trace:
I20260320 15:27:27.941345 28730 rpcz_store.cc:276] 0320 15:27:25.553830 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.553897 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:27.938690 (+2384793us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:27.938713 (+ 23us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.938835 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2384438}
I20260320 15:27:27.942423 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 295) took 1961 ms. Trace:
I20260320 15:27:27.931581 29233 raft_consensus.cc:515] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.942564 28730 rpcz_store.cc:276] 0320 15:27:25.980311 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.983597 (+ 3286us) service_pool.cc:225] Handling call
0320 15:27:27.941670 (+1958073us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:27.941683 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.941798 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1957826}
I20260320 15:27:27.943986 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 296) took 1963 ms. Trace:
I20260320 15:27:27.944108 28730 rpcz_store.cc:276] 0320 15:27:25.980430 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.983892 (+ 3462us) service_pool.cc:225] Handling call
0320 15:27:27.943842 (+1959950us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:27.943852 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.943955 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1959742}
I20260320 15:27:27.944442 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 297) took 1963 ms. Trace:
I20260320 15:27:27.944546 28730 rpcz_store.cc:276] 0320 15:27:25.980523 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.984121 (+ 3598us) service_pool.cc:225] Handling call
0320 15:27:27.944334 (+1960213us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:27.944341 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.944411 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1960056}
I20260320 15:27:27.944936 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 298) took 1964 ms. Trace:
I20260320 15:27:27.944943 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:27.945052 28730 rpcz_store.cc:276] 0320 15:27:25.980689 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.984311 (+ 3622us) service_pool.cc:225] Handling call
0320 15:27:27.944736 (+1960425us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:27.944801 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.944896 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7424,"tablet-delete.queue_time_us":1960218}
I20260320 15:27:27.945375 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 299) took 1964 ms. Trace:
I20260320 15:27:27.945470 28730 rpcz_store.cc:276] 0320 15:27:25.980782 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.984563 (+ 3781us) service_pool.cc:225] Handling call
0320 15:27:27.945270 (+1960707us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:27.945278 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.945349 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1960508}
I20260320 15:27:27.945748 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 300) took 1964 ms. Trace:
I20260320 15:27:27.945854 28730 rpcz_store.cc:276] 0320 15:27:25.980864 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.984797 (+ 3933us) service_pool.cc:225] Handling call
0320 15:27:27.945653 (+1960856us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:27.945660 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.945722 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1960695}
I20260320 15:27:27.946120 28730 tablet_replica.cc:333] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:27.946488 28730 raft_consensus.cc:2243] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:27.946712 28730 raft_consensus.cc:2272] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:27.932308 28735 log.cc:1199] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b8f3ee3c611d41b483bd35f868042d05
W20260320 15:27:27.946741 28239 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:27.947114 29233 leader_election.cc:290] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:27.949461 28735 ts_tablet_manager.cc:1950] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:27.950037 27944 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskc3SESu/build/asan/bin/kudu pid 28020
I20260320 15:27:27.932333 28049 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet 7be03035dc3d44f9aa5b43b49281a940
W20260320 15:27:27.933568 28041 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ba4eb20eb0f1403282030727757031f7 consensus state has no leader
I20260320 15:27:27.951824 28041 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]) successfully deleted
I20260320 15:27:27.954031 28049 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:34644:
table { table_name: "default.Table_818e1205d4f34148aebe2a4206ca41a5" } new_table_name: "default.Table_a9b80338873049fbaaa6c166ee20099e" modify_external_catalogs: true
W20260320 15:27:27.957525 28045 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet db16d86664794305bde714a7ff4d5caf has no consensus state
I20260320 15:27:27.957937 28048 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Sending DeleteTablet for 3 replicas of tablet ffe278739bbc4ddab848af9c25241380
W20260320 15:27:27.959163 28047 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06cb44b9cc6445cf8324cf52190d6773 consensus state has no leader
I20260320 15:27:27.964047 28084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260320 15:27:27.970397 28730 ts_tablet_manager.cc:1916] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:27.988238 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 303) took 2702 ms. Trace:
I20260320 15:27:27.988423 28735 rpcz_store.cc:276] 0320 15:27:25.285877 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.286810 (+ 933us) service_pool.cc:225] Handling call
0320 15:27:27.677936 (+2391126us) ts_tablet_manager.cc:1151] Deleting tablet b8f3ee3c611d41b483bd35f868042d05
0320 15:27:27.677950 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.698935 (+ 20985us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x616000162a54. stack: 00007f7f40429076 00007f7f40428e30 00007f7f3f3e33ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f7f4c3287e0 00007f7f4d679dc3 00007f7f5023da77 00007f7f50270b44 00007f7f50251560 00007f7f502512ed 0000000000a8b58c 00007f7f4046ef5f 00007f7f4047227b 00007f7f404720f8
0320 15:27:27.792029 (+ 93094us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.811417 (+ 19388us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:27.988144 (+176727us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":47733,"spinlock_wait_cycles":46507264,"tablet-delete.queue_time_us":2390832}
I20260320 15:27:27.932955 28736 raft_consensus.cc:385] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:27.989122 28041 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b8f3ee3c611d41b483bd35f868042d05 (table default.Table_78dd3a47806c44f186bb967be20e8d9f [id=79ab13b2c3fa42b4b744700487fd8b8d]) successfully deleted
I20260320 15:27:27.957633 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.989214 28736 raft_consensus.cc:740] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:27.961262 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.961237 28282 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:57830
I20260320 15:27:27.989924 28736 consensus_queue.cc:260] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:27.990869 28736 ts_tablet_manager.cc:1434] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.173s user 0.004s sys 0.000s
I20260320 15:27:27.993242 28382 leader_election.cc:304] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.006182 28736 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.015650 28735 tablet_replica.cc:333] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.015885 28733 tablet_bootstrap.cc:492] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.016108 28733 ts_tablet_manager.cc:1403] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.003s
I20260320 15:27:28.016337 28735 raft_consensus.cc:2243] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.016619 28735 raft_consensus.cc:2272] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.017146 28733 raft_consensus.cc:359] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.017369 28733 raft_consensus.cc:385] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.017277 28735 ts_tablet_manager.cc:1916] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.017494 28733 raft_consensus.cc:740] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.017767 28733 consensus_queue.cc:260] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.018286 28733 ts_tablet_manager.cc:1434] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:28.019429 28733 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.022831 28730 ts_tablet_manager.cc:1929] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.023012 28730 log.cc:1199] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6d1db9963c4645ae9aab72b4b22d6938
I20260320 15:27:28.023415 28730 ts_tablet_manager.cc:1950] T 6d1db9963c4645ae9aab72b4b22d6938 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:28.023854 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 304) took 1789 ms. Trace:
I20260320 15:27:28.023970 28730 rpcz_store.cc:276] 0320 15:27:26.234177 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.234293 (+ 116us) service_pool.cc:225] Handling call
0320 15:27:27.946043 (+1711750us) ts_tablet_manager.cc:1151] Deleting tablet 6d1db9963c4645ae9aab72b4b22d6938
0320 15:27:27.946057 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:27.999958 (+ 53901us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.022688 (+ 22730us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.023787 (+ 1099us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1681622}
I20260320 15:27:28.024369 28730 tablet_replica.cc:333] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:28.024585 28730 raft_consensus.cc:2243] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.024794 28730 raft_consensus.cc:2272] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.025453 28730 ts_tablet_manager.cc:1916] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.050366 28736 tablet_bootstrap.cc:654] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.051869 28733 tablet_bootstrap.cc:654] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.059923 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:28.062449 28733 tablet_bootstrap.cc:492] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.062657 28733 ts_tablet_manager.cc:1403] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260320 15:27:28.063510 28733 raft_consensus.cc:359] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.063768 28733 raft_consensus.cc:385] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.063856 28733 raft_consensus.cc:740] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.064136 28733 consensus_queue.cc:260] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.064400 28729 tablet_replica.cc:333] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.064728 28733 ts_tablet_manager.cc:1434] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:28.068502 29254 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 07e8bb8819eb4cb4892e0cba1f758892)
I20260320 15:27:28.068727 29254 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:28.070171 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 5 candidate_status { last_received { term: 4 index: 537 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:28.072806 29254 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
W20260320 15:27:28.073560 29239 log.cc:927] Time spent T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
W20260320 15:27:28.073768 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 112720us
W20260320 15:27:28.074695 28240 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:28.074892 28240 leader_election.cc:304] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.075424 29233 raft_consensus.cc:2749] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.075630 29156 raft_consensus.cc:2749] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.075812 28444 raft_consensus.cc:2468] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.082425 29197 consensus_queue.cc:1048] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:28.083585 28382 consensus_peers.cc:597] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f591c45d9d73442ab1f0144ddfc8c48b. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:28.085036 28240 leader_election.cc:304] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
W20260320 15:27:28.085417 28240 raft_consensus.cc:2686] T b8f3ee3c611d41b483bd35f868042d05 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:28.086781 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:28.086959 28729 raft_consensus.cc:2243] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.098646 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:39582
I20260320 15:27:28.101337 28733 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.103513 28733 tablet_bootstrap.cc:654] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.104545 28730 ts_tablet_manager.cc:1929] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.104699 28730 log.cc:1199] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:28.105144 28730 ts_tablet_manager.cc:1950] T 5ada1f644b754fbe8b3790b46ec8cdf4 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:28.106043 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 307) took 1837 ms. Trace:
I20260320 15:27:28.106192 28730 rpcz_store.cc:276] 0320 15:27:26.268727 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.268799 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:28.024266 (+1755467us) ts_tablet_manager.cc:1151] Deleting tablet 5ada1f644b754fbe8b3790b46ec8cdf4
0320 15:27:28.024276 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.083685 (+ 59409us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.087385 (+ 3700us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.105991 (+ 18606us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1754614}
I20260320 15:27:28.106690 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 308) took 1834 ms. Trace:
I20260320 15:27:28.106807 28730 rpcz_store.cc:276] 0320 15:27:26.272463 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.272534 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:28.106514 (+1833980us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:28.106527 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.106647 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1830625}
I20260320 15:27:28.107203 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 309) took 1829 ms. Trace:
I20260320 15:27:28.107321 28730 rpcz_store.cc:276] 0320 15:27:26.277322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.277386 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:28.107068 (+1829682us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:28.107076 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.107163 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1810406}
I20260320 15:27:28.108547 28736 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.108716 28736 ts_tablet_manager.cc:1403] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.103s user 0.003s sys 0.002s
I20260320 15:27:28.109731 28736 raft_consensus.cc:359] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.110013 28736 raft_consensus.cc:385] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.110090 28736 raft_consensus.cc:740] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.110415 28736 consensus_queue.cc:260] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.111719 28729 raft_consensus.cc:2272] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.112422 28729 ts_tablet_manager.cc:1916] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.108438 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.113730 28447 raft_consensus.cc:2468] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:28.115999 28240 leader_election.cc:304] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.118191 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 315) took 1783 ms. Trace:
I20260320 15:27:28.118359 28730 rpcz_store.cc:276] 0320 15:27:26.335159 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.335210 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:28.117973 (+1782763us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:28.117992 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.118138 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1772042}
I20260320 15:27:28.118855 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 316) took 1778 ms. Trace:
I20260320 15:27:28.118983 28730 rpcz_store.cc:276] 0320 15:27:26.340718 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.340787 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:28.118690 (+1777903us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:28.118702 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.118818 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1777557}
I20260320 15:27:28.119190 28735 ts_tablet_manager.cc:1929] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.119338 28735 log.cc:1199] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f591c45d9d73442ab1f0144ddfc8c48b
I20260320 15:27:28.119359 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 317) took 1778 ms. Trace:
I20260320 15:27:28.119462 28730 rpcz_store.cc:276] 0320 15:27:26.340897 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.341188 (+ 291us) service_pool.cc:225] Handling call
0320 15:27:28.119242 (+1778054us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:28.119251 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.119330 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1777814}
I20260320 15:27:28.119848 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 318) took 1778 ms. Trace:
I20260320 15:27:28.125535 28733 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.125741 28733 ts_tablet_manager.cc:1403] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.001s
I20260320 15:27:28.126623 28733 raft_consensus.cc:359] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.126847 28733 raft_consensus.cc:385] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.126931 28733 raft_consensus.cc:740] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.127178 28733 consensus_queue.cc:260] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.127808 28733 ts_tablet_manager.cc:1434] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:28.128193 28733 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.130844 28733 tablet_bootstrap.cc:654] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.132169 29233 raft_consensus.cc:2804] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.132360 29233 raft_consensus.cc:493] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.132477 29233 raft_consensus.cc:3060] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.119951 28730 rpcz_store.cc:276] 0320 15:27:26.341019 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.341453 (+ 434us) service_pool.cc:225] Handling call
0320 15:27:28.119720 (+1778267us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:28.119728 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.119814 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1777928}
I20260320 15:27:28.135345 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 323) took 1481 ms. Trace:
I20260320 15:27:28.135646 28730 rpcz_store.cc:276] 0320 15:27:26.653975 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.654300 (+ 325us) service_pool.cc:225] Handling call
0320 15:27:28.135153 (+1480853us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:28.135165 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.135297 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1478395}
I20260320 15:27:28.136111 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 325) took 1440 ms. Trace:
I20260320 15:27:28.136238 28730 rpcz_store.cc:276] 0320 15:27:26.695881 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.696059 (+ 178us) service_pool.cc:225] Handling call
0320 15:27:28.135938 (+1439879us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:28.135945 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.136068 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1364908}
I20260320 15:27:28.136608 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 326) took 1440 ms. Trace:
I20260320 15:27:28.136718 28730 rpcz_store.cc:276] 0320 15:27:26.695997 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.696236 (+ 239us) service_pool.cc:225] Handling call
0320 15:27:28.136454 (+1440218us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:28.136464 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.136562 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1365355}
I20260320 15:27:28.137089 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 324) took 1441 ms. Trace:
I20260320 15:27:28.137203 28730 rpcz_store.cc:276] 0320 15:27:26.695690 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.732691 (+ 37001us) service_pool.cc:225] Handling call
0320 15:27:28.136954 (+1404263us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:28.136965 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.137048 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1358585}
I20260320 15:27:28.137538 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 328) took 1026 ms. Trace:
I20260320 15:27:28.137645 28730 rpcz_store.cc:276] 0320 15:27:27.111163 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.111891 (+ 728us) service_pool.cc:225] Handling call
0320 15:27:28.137420 (+1025529us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:28.137428 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.137506 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1025110}
I20260320 15:27:28.137941 28730 tablet_replica.cc:333] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:28.138154 28730 raft_consensus.cc:2243] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.138487 28730 raft_consensus.cc:2272] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.139941 29233 raft_consensus.cc:515] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.141733 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:28.142540 28239 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.142935 29233 leader_election.cc:290] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.143728 28736 ts_tablet_manager.cc:1434] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.035s user 0.001s sys 0.002s
I20260320 15:27:28.144197 28736 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.150099 28730 ts_tablet_manager.cc:1916] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:28.154834 28383 consensus_peers.cc:597] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:28.156380 28736 tablet_bootstrap.cc:654] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.157251 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:28.160261 29248 log.cc:927] Time spent T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260320 15:27:28.161307 28447 raft_consensus.cc:3060] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.166689 28733 tablet_bootstrap.cc:492] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.166989 28733 ts_tablet_manager.cc:1403] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.000s
I20260320 15:27:28.168020 28733 raft_consensus.cc:359] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.168257 28733 raft_consensus.cc:385] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.168344 28733 raft_consensus.cc:740] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.168638 28733 consensus_queue.cc:260] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.169195 28733 ts_tablet_manager.cc:1434] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:28.169603 28733 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.171835 28733 tablet_bootstrap.cc:654] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.179925 28447 raft_consensus.cc:2468] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.180523 28736 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.180687 28240 leader_election.cc:304] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.180809 28736 ts_tablet_manager.cc:1403] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.006s
I20260320 15:27:28.181213 29233 raft_consensus.cc:2804] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.181378 29233 raft_consensus.cc:697] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:28.181607 28730 ts_tablet_manager.cc:1929] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.181779 28730 log.cc:1199] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2ca38bc990634e21a2c38dd840488f8c
I20260320 15:27:28.181687 29233 consensus_queue.cc:237] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.182243 28730 ts_tablet_manager.cc:1950] T 2ca38bc990634e21a2c38dd840488f8c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:28.183118 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 330) took 1071 ms. Trace:
I20260320 15:27:28.183234 28730 rpcz_store.cc:276] 0320 15:27:27.111810 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.112064 (+ 254us) service_pool.cc:225] Handling call
0320 15:27:28.137877 (+1025813us) ts_tablet_manager.cc:1151] Deleting tablet 2ca38bc990634e21a2c38dd840488f8c
0320 15:27:28.137885 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.153729 (+ 15844us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.181451 (+ 27722us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.183056 (+ 1605us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1025330}
I20260320 15:27:28.183758 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 329) took 1072 ms. Trace:
I20260320 15:27:28.183862 28730 rpcz_store.cc:276] 0320 15:27:27.111310 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.112028 (+ 718us) service_pool.cc:225] Handling call
0320 15:27:28.183573 (+1071545us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:28.183584 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.183703 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1070925}
I20260320 15:27:28.184242 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 332) took 1070 ms. Trace:
I20260320 15:27:28.184333 28730 rpcz_store.cc:276] 0320 15:27:27.113782 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.114074 (+ 292us) service_pool.cc:225] Handling call
0320 15:27:28.184088 (+1070014us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:28.184097 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.184179 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1069719}
I20260320 15:27:28.187039 28736 raft_consensus.cc:359] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.187975 28736 raft_consensus.cc:385] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.188511 28736 raft_consensus.cc:740] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.189126 28736 consensus_queue.cc:260] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.190290 28736 ts_tablet_manager.cc:1434] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260320 15:27:28.191161 28736 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.193820 28736 tablet_bootstrap.cc:654] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.199123 28733 tablet_bootstrap.cc:492] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.199446 28733 ts_tablet_manager.cc:1403] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260320 15:27:28.200426 28729 ts_tablet_manager.cc:1929] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.200630 28729 log.cc:1199] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:28.200933 28733 raft_consensus.cc:359] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.201164 28733 raft_consensus.cc:385] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.201290 28733 raft_consensus.cc:740] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.201572 28733 consensus_queue.cc:260] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.202216 28733 ts_tablet_manager.cc:1434] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260320 15:27:28.202339 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 331) took 1071 ms. Trace:
I20260320 15:27:28.202448 28730 rpcz_store.cc:276] 0320 15:27:27.113673 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.114418 (+ 745us) service_pool.cc:225] Handling call
0320 15:27:28.184563 (+1070145us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:28.184571 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.184666 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1069958}
I20260320 15:27:28.203127 28733 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.205437 28733 tablet_bootstrap.cc:654] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.209105 28730 tablet_replica.cc:333] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:28.209367 28730 raft_consensus.cc:2243] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.209609 28730 raft_consensus.cc:2272] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.210964 28730 ts_tablet_manager.cc:1916] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.219632 28729 ts_tablet_manager.cc:1950] T b8f3ee3c611d41b483bd35f868042d05 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.221022 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 303) took 2773 ms. Trace:
I20260320 15:27:28.221181 28736 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.221267 28729 rpcz_store.cc:276] 0320 15:27:25.447391 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.447892 (+ 501us) service_pool.cc:225] Handling call
0320 15:27:27.932857 (+2484965us) ts_tablet_manager.cc:1151] Deleting tablet b8f3ee3c611d41b483bd35f868042d05
0320 15:27:27.932874 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.108822 (+175948us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x6160000f2854. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff3892697e0 00007ff38a5badc3 00007ff38d17ea77 00007ff38d1b1b44 00007ff38d192560 00007ff38d1922ed 0000000000a8b58c 00007ff37d3aff5f 00007ff37d3b327b 00007ff37d3b30f8
0320 15:27:28.190635 (+ 81813us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.200193 (+ 9558us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.220976 (+ 20783us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9818,"spinlock_wait_cycles":8542464,"tablet-delete.queue_time_us":2436909}
I20260320 15:27:28.221544 28736 ts_tablet_manager.cc:1403] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.004s
I20260320 15:27:28.222301 28729 tablet_replica.cc:333] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.222646 28729 raft_consensus.cc:2243] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.222769 28736 raft_consensus.cc:359] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.223093 28729 pending_rounds.cc:70] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:28.223106 28736 raft_consensus.cc:385] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.223191 28736 raft_consensus.cc:740] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.223304 28729 pending_rounds.cc:77] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387749877018624 op_type: NO_OP noop_request { }
I20260320 15:27:28.223778 28736 consensus_queue.cc:260] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.224926 28735 ts_tablet_manager.cc:1950] T f591c45d9d73442ab1f0144ddfc8c48b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.227808 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 304) took 2883 ms. Trace:
I20260320 15:27:28.227968 28735 rpcz_store.cc:276] 0320 15:27:25.344395 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.344775 (+ 380us) service_pool.cc:225] Handling call
0320 15:27:28.015512 (+2670737us) ts_tablet_manager.cc:1151] Deleting tablet f591c45d9d73442ab1f0144ddfc8c48b
0320 15:27:28.015550 (+ 38us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.115311 (+ 99761us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.119030 (+ 3719us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.227753 (+108723us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2658367}
I20260320 15:27:28.228391 28735 tablet_replica.cc:333] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.228611 28736 ts_tablet_manager.cc:1434] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:28.230254 28736 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.231880 28733 tablet_bootstrap.cc:492] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.232079 28733 ts_tablet_manager.cc:1403] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.000s
I20260320 15:27:28.233080 28736 tablet_bootstrap.cc:654] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.233233 28733 raft_consensus.cc:359] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.233479 28733 raft_consensus.cc:385] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.233577 28733 raft_consensus.cc:740] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.233870 28733 consensus_queue.cc:260] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.234927 28733 ts_tablet_manager.cc:1434] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260320 15:27:28.237007 28733 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.238430 28729 raft_consensus.cc:2889] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:28.238718 28729 raft_consensus.cc:2272] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.239075 28733 tablet_bootstrap.cc:654] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.239641 28729 ts_tablet_manager.cc:1916] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.243075 28735 raft_consensus.cc:2243] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.260196 28730 ts_tablet_manager.cc:1929] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.266114 28730 log.cc:1199] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/557d8be0e4bf42239b2bb4585e7d0f06
I20260320 15:27:28.256910 29233 raft_consensus.cc:493] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.267133 28730 ts_tablet_manager.cc:1950] T 557d8be0e4bf42239b2bb4585e7d0f06 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:28.267195 28735 raft_consensus.cc:2272] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.267195 29233 raft_consensus.cc:515] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.267938 28735 ts_tablet_manager.cc:1916] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.268142 29233 leader_election.cc:290] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.269052 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.269315 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:28.269737 28240 leader_election.cc:343] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:28.270548 28239 leader_election.cc:343] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.270704 28239 leader_election.cc:304] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.272293 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 333) took 1158 ms. Trace:
I20260320 15:27:28.274202 28730 rpcz_store.cc:276] 0320 15:27:27.113900 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.114725 (+ 825us) service_pool.cc:225] Handling call
0320 15:27:28.202816 (+1088091us) ts_tablet_manager.cc:1151] Deleting tablet 557d8be0e4bf42239b2bb4585e7d0f06
0320 15:27:28.208985 (+ 6169us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.222303 (+ 13318us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.260016 (+ 37713us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.272246 (+ 12230us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1087883}
I20260320 15:27:28.276633 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 337) took 1122 ms. Trace:
I20260320 15:27:28.276877 28730 rpcz_store.cc:276] 0320 15:27:27.154039 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.154118 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:28.276432 (+1122314us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:28.276447 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.276589 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1121753}
I20260320 15:27:28.279090 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 338) took 1124 ms. Trace:
I20260320 15:27:28.266188 28734 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.281299 28734 ts_tablet_manager.cc:1403] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.923s user 0.023s sys 0.038s
I20260320 15:27:28.282085 28734 raft_consensus.cc:359] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.282287 28734 raft_consensus.cc:385] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.282370 28734 raft_consensus.cc:740] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.282605 28734 consensus_queue.cc:260] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.282958 29197 raft_consensus.cc:493] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.282899 28733 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.283110 29197 raft_consensus.cc:515] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.283298 28733 ts_tablet_manager.cc:1403] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.000s
I20260320 15:27:28.283854 29197 leader_election.cc:290] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:28.284641 28733 raft_consensus.cc:359] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.284752 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.284916 28733 raft_consensus.cc:385] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.285045 28733 raft_consensus.cc:740] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.285378 28733 consensus_queue.cc:260] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.286078 28733 ts_tablet_manager.cc:1434] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260320 15:27:28.283056 28730 rpcz_store.cc:276] 0320 15:27:27.154499 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.154573 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:28.278903 (+1124330us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:28.278916 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.279043 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1124099}
I20260320 15:27:28.298946 28733 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.299510 28734 ts_tablet_manager.cc:1434] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260320 15:27:28.303383 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 341) took 1094 ms. Trace:
I20260320 15:27:28.306319 28730 rpcz_store.cc:276] 0320 15:27:27.208765 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.208854 (+ 89us) service_pool.cc:225] Handling call
0320 15:27:28.289975 (+1081121us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:28.302517 (+ 12542us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.302835 (+ 318us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1080816}
I20260320 15:27:28.307104 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 342) took 1066 ms. Trace:
I20260320 15:27:28.309211 28730 rpcz_store.cc:276] 0320 15:27:27.240948 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.246763 (+ 5815us) service_pool.cc:225] Handling call
0320 15:27:28.306928 (+1060165us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:28.306941 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.307071 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1047247}
I20260320 15:27:28.310010 28730 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51060 (request call id 343) took 1012 ms. Trace:
I20260320 15:27:28.311504 28730 rpcz_store.cc:276] 0320 15:27:27.297120 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.297198 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:28.309857 (+1012659us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:28.309869 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.309979 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1012250}
I20260320 15:27:28.311877 28734 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.310614 28733 tablet_bootstrap.cc:654] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.318274 28729 ts_tablet_manager.cc:1929] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.319653 28729 log.cc:1199] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f591c45d9d73442ab1f0144ddfc8c48b
I20260320 15:27:28.320236 28729 ts_tablet_manager.cc:1950] T f591c45d9d73442ab1f0144ddfc8c48b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.321153 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 305) took 2822 ms. Trace:
I20260320 15:27:28.319506 28734 tablet_bootstrap.cc:654] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.330389 29246 consensus_queue.cc:1048] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:28.354637 29233 raft_consensus.cc:2749] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.356206 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b4040a0214a46b580b2ac2de59243ce" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.356526 28297 raft_consensus.cc:2393] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:28.357285 29245 consensus_queue.cc:1048] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:28.358685 28524 leader_election.cc:304] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.359375 29197 raft_consensus.cc:3060] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.362470 29197 raft_consensus.cc:2749] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.370925 28729 rpcz_store.cc:276] 0320 15:27:25.498687 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.498795 (+ 108us) service_pool.cc:225] Handling call
0320 15:27:28.222190 (+2723395us) ts_tablet_manager.cc:1151] Deleting tablet f591c45d9d73442ab1f0144ddfc8c48b
0320 15:27:28.222206 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.278768 (+ 56562us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.318098 (+ 39330us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.321111 (+ 3013us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2703852}
I20260320 15:27:28.372092 28729 tablet_replica.cc:333] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.375633 28736 tablet_bootstrap.cc:492] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.375833 28736 ts_tablet_manager.cc:1403] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.146s user 0.011s sys 0.037s
I20260320 15:27:28.376362 28733 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.376665 28733 ts_tablet_manager.cc:1403] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.078s user 0.002s sys 0.005s
I20260320 15:27:28.377662 29197 raft_consensus.cc:493] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.377790 28733 raft_consensus.cc:359] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.378017 28733 raft_consensus.cc:385] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.377925 29197 raft_consensus.cc:515] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.378103 28733 raft_consensus.cc:740] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.378665 28733 consensus_queue.cc:260] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.380715 29197 leader_election.cc:290] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.382072 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.382839 28736 raft_consensus.cc:359] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.383071 28736 raft_consensus.cc:385] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:28.383050 28524 leader_election.cc:343] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.383193 28736 raft_consensus.cc:740] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.383476 28736 consensus_queue.cc:260] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.384176 28736 ts_tablet_manager.cc:1434] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260320 15:27:28.384550 28736 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.386695 28736 tablet_bootstrap.cc:654] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.380110 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.389514 28444 raft_consensus.cc:2468] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:28.390254 28524 leader_election.cc:304] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:28.390777 29197 raft_consensus.cc:2804] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.390874 29197 raft_consensus.cc:493] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.390974 29197 raft_consensus.cc:3060] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.393795 29237 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.393971 29237 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.394256 29197 raft_consensus.cc:515] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.394758 29237 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:28.394824 29197 leader_election.cc:290] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.395052 29233 raft_consensus.cc:493] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.395377 28735 ts_tablet_manager.cc:1929] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.395259 29233 raft_consensus.cc:515] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.395942 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.396023 29233 leader_election.cc:290] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.396226 28444 raft_consensus.cc:3060] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.397974 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.398239 28446 raft_consensus.cc:2468] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:28.398754 28240 leader_election.cc:304] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.399220 29233 raft_consensus.cc:2804] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.399339 29233 raft_consensus.cc:493] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.399470 29233 raft_consensus.cc:3060] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.400094 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.402045 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c2339ff2c474a288876f32b85cecc72" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.402691 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcec6175c1ca49f5b682a29d5c1e235a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:28.402770 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.402894 28303 raft_consensus.cc:3060] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.402482 28297 raft_consensus.cc:2393] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:28.402997 29233 raft_consensus.cc:515] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.404250 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.404491 28447 raft_consensus.cc:3060] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.406177 28303 raft_consensus.cc:2468] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:28.407127 28239 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.407358 28447 raft_consensus.cc:2468] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.407694 28524 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.407959 28240 leader_election.cc:304] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.408311 29241 raft_consensus.cc:2804] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.408309 28524 leader_election.cc:304] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:28.408594 29237 raft_consensus.cc:3060] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.412662 29233 leader_election.cc:290] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.412719 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:28.413604 28239 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.413846 29237 raft_consensus.cc:2749] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.414134 29237 raft_consensus.cc:2804] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.414301 29237 raft_consensus.cc:697] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:28.414629 29237 consensus_queue.cc:237] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.415365 28735 log.cc:1199] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/d547c032332d4d109222b4743fa28561
I20260320 15:27:28.399499 28444 raft_consensus.cc:2468] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:28.416009 28735 ts_tablet_manager.cc:1950] T d547c032332d4d109222b4743fa28561 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.417006 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 305) took 3054 ms. Trace:
I20260320 15:27:28.417138 28735 rpcz_store.cc:276] 0320 15:27:25.362621 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.362777 (+ 156us) service_pool.cc:225] Handling call
0320 15:27:28.228299 (+2865522us) ts_tablet_manager.cc:1151] Deleting tablet d547c032332d4d109222b4743fa28561
0320 15:27:28.228311 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.391964 (+163653us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.395220 (+ 3256us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.416941 (+ 21721us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14364,"tablet-delete.queue_time_us":2864682}
I20260320 15:27:28.417591 28735 tablet_replica.cc:333] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.417896 28735 raft_consensus.cc:2243] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.418196 28735 raft_consensus.cc:2272] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.424444 29241 raft_consensus.cc:697] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:28.425026 29241 consensus_queue.cc:237] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.427214 28736 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.427423 28736 ts_tablet_manager.cc:1403] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.003s
I20260320 15:27:28.428337 28736 raft_consensus.cc:359] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.428606 28736 raft_consensus.cc:385] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.428717 28736 raft_consensus.cc:740] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.429000 28736 consensus_queue.cc:260] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.429767 28736 ts_tablet_manager.cc:1434] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:28.430152 28736 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.431600 28729 raft_consensus.cc:2243] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.432006 28729 pending_rounds.cc:70] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:28.432202 28729 pending_rounds.cc:77] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387753121759232 op_type: NO_OP noop_request { }
I20260320 15:27:28.432426 28729 raft_consensus.cc:2889] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:28.432613 28729 raft_consensus.cc:2272] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.432796 28735 ts_tablet_manager.cc:1916] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.433581 28729 ts_tablet_manager.cc:1916] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.442463 28733 ts_tablet_manager.cc:1434] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.066s user 0.002s sys 0.000s
I20260320 15:27:28.449303 29275 raft_consensus.cc:493] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.449649 28736 tablet_bootstrap.cc:654] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.449550 29275 raft_consensus.cc:515] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.450479 29275 leader_election.cc:290] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.451830 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.451819 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.452054 28444 raft_consensus.cc:2468] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:28.452714 28240 leader_election.cc:304] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:28.453398 28239 leader_election.cc:343] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.453830 29275 raft_consensus.cc:2804] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.453945 29275 raft_consensus.cc:493] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.454036 29275 raft_consensus.cc:3060] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.454023 28733 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.456032 28733 tablet_bootstrap.cc:654] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.458294 28734 tablet_bootstrap.cc:492] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.458511 28734 ts_tablet_manager.cc:1403] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.147s user 0.002s sys 0.005s
I20260320 15:27:28.459412 28734 raft_consensus.cc:359] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.459667 28734 raft_consensus.cc:385] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.459739 28734 raft_consensus.cc:740] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.460016 28734 consensus_queue.cc:260] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.460534 29275 raft_consensus.cc:515] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.462381 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.462689 28444 raft_consensus.cc:3060] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.463582 28736 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.463752 28736 ts_tablet_manager.cc:1403] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.001s
I20260320 15:27:28.464654 28736 raft_consensus.cc:359] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.464869 28736 raft_consensus.cc:385] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.464957 28736 raft_consensus.cc:740] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.465191 28736 consensus_queue.cc:260] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.465898 28736 ts_tablet_manager.cc:1434] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:28.467381 28444 raft_consensus.cc:2468] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.467851 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:28.468034 29275 leader_election.cc:290] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:28.468724 28239 leader_election.cc:343] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.469323 28240 leader_election.cc:304] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.470140 29275 raft_consensus.cc:2804] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.470613 29275 raft_consensus.cc:697] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:28.471079 29275 consensus_queue.cc:237] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.466426 28736 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.467625 28734 ts_tablet_manager.cc:1434] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:28.473035 29280 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 07e8bb8819eb4cb4892e0cba1f758892)
I20260320 15:27:28.473347 29280 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:28.479205 28734 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.480902 29280 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:28.481325 28734 tablet_bootstrap.cc:654] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.481323 28736 tablet_bootstrap.cc:654] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.484328 28735 ts_tablet_manager.cc:1929] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.485780 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 5 candidate_status { last_received { term: 4 index: 547 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:28.486034 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 4.
I20260320 15:27:28.486600 28872 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters:
I20260320 15:27:28.486974 28735 log.cc:1199] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/557d8be0e4bf42239b2bb4585e7d0f06
I20260320 15:27:28.487113 29280 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260320 15:27:28.487339 29280 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 07e8bb8819eb4cb4892e0cba1f758892)
I20260320 15:27:28.487576 29280 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 4 FOLLOWER]: Advancing to term 5
I20260320 15:27:28.487856 28735 ts_tablet_manager.cc:1950] T 557d8be0e4bf42239b2bb4585e7d0f06 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.490499 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 306) took 3085 ms. Trace:
I20260320 15:27:28.490679 28735 rpcz_store.cc:276] 0320 15:27:25.404946 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.405027 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:28.417483 (+3012456us) ts_tablet_manager.cc:1151] Deleting tablet 557d8be0e4bf42239b2bb4585e7d0f06
0320 15:27:28.417497 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.446397 (+ 28900us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.484134 (+ 37737us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.490425 (+ 6291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3002092}
I20260320 15:27:28.491398 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 307) took 2802 ms. Trace:
I20260320 15:27:28.491595 28735 rpcz_store.cc:276] 0320 15:27:25.688902 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.689268 (+ 366us) service_pool.cc:225] Handling call
0320 15:27:28.491213 (+2801945us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:28.491224 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.491351 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2783547}
I20260320 15:27:28.492540 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 312) took 2584 ms. Trace:
I20260320 15:27:28.492689 28735 rpcz_store.cc:276] 0320 15:27:25.907796 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.907845 (+ 49us) service_pool.cc:225] Handling call
0320 15:27:28.492376 (+2584531us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:28.492387 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.492499 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2584285}
I20260320 15:27:28.492882 28733 tablet_bootstrap.cc:492] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.493045 28733 ts_tablet_manager.cc:1403] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.000s
I20260320 15:27:28.493299 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 311) took 2585 ms. Trace:
I20260320 15:27:28.493259 29280 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:28.493464 28735 rpcz_store.cc:276] 0320 15:27:25.907373 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.918864 (+ 11491us) service_pool.cc:225] Handling call
0320 15:27:28.493107 (+2574243us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:28.493117 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.493257 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2573999}
I20260320 15:27:28.493959 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 313) took 2553 ms. Trace:
I20260320 15:27:28.493955 28733 raft_consensus.cc:359] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.494145 28733 raft_consensus.cc:385] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.494086 28735 rpcz_store.cc:276] 0320 15:27:25.940930 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.941304 (+ 374us) service_pool.cc:225] Handling call
0320 15:27:28.493826 (+2552522us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:28.493834 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.493915 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2552217}
I20260320 15:27:28.494230 28733 raft_consensus.cc:740] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.494557 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 316) took 2550 ms. Trace:
I20260320 15:27:28.494526 28733 consensus_queue.cc:260] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.494679 28735 rpcz_store.cc:276] 0320 15:27:25.943919 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.943980 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:28.494402 (+2550422us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:28.494410 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.494518 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2550103}
I20260320 15:27:28.495340 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 315) took 2552 ms. Trace:
I20260320 15:27:28.495391 29280 leader_election.cc:290] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 election: Requested vote from peers 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:28.495478 28735 rpcz_store.cc:276] 0320 15:27:25.943100 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.944762 (+ 1662us) service_pool.cc:225] Handling call
0320 15:27:28.495213 (+2550451us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:28.495222 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.495301 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2550186}
I20260320 15:27:28.496054 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 314) took 2554 ms. Trace:
I20260320 15:27:28.496227 28735 rpcz_store.cc:276] 0320 15:27:25.941906 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.945380 (+ 3474us) service_pool.cc:225] Handling call
0320 15:27:28.495876 (+2550496us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:28.495890 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.496020 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2550277}
I20260320 15:27:28.496819 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 322) took 2303 ms. Trace:
I20260320 15:27:28.496986 28735 rpcz_store.cc:276] 0320 15:27:26.193204 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.193285 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:28.496702 (+2303417us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:28.496711 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.496787 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2303141}
I20260320 15:27:28.497025 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" candidate_term: 5 candidate_status { last_received { term: 4 index: 547 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a"
I20260320 15:27:28.497707 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 323) took 2293 ms. Trace:
I20260320 15:27:28.497880 28735 rpcz_store.cc:276] 0320 15:27:26.203916 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.204004 (+ 88us) service_pool.cc:225] Handling call
0320 15:27:28.497550 (+2293546us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:28.497565 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.497667 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2285119}
I20260320 15:27:28.498317 28736 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.498538 28736 ts_tablet_manager.cc:1403] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.002s
I20260320 15:27:28.498579 28735 tablet_replica.cc:333] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.498963 28735 raft_consensus.cc:2243] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.499265 28735 raft_consensus.cc:2272] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.499665 28736 raft_consensus.cc:359] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.503109 28134 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 4 FOLLOWER]: Advancing to term 5
I20260320 15:27:28.505404 28735 ts_tablet_manager.cc:1916] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.506459 28134 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1caebf640d4ec0bf9b28c4f496ca18 in term 5.
I20260320 15:27:28.507066 28872 leader_election.cc:304] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18; no voters:
I20260320 15:27:28.507652 28733 ts_tablet_manager.cc:1434] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.014s user 0.002s sys 0.001s
I20260320 15:27:28.507666 29280 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 5 FOLLOWER]: Leader election won for term 5
I20260320 15:27:28.509232 28733 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.511307 28733 tablet_bootstrap.cc:654] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.512229 28736 raft_consensus.cc:385] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.512346 28736 raft_consensus.cc:740] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.513170 28736 consensus_queue.cc:260] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.514629 28736 ts_tablet_manager.cc:1434] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260320 15:27:28.515125 28736 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.531018 28734 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.531212 28734 ts_tablet_manager.cc:1403] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.003s
I20260320 15:27:28.532114 28734 raft_consensus.cc:359] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.532328 28734 raft_consensus.cc:385] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.532428 28734 raft_consensus.cc:740] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.532721 28734 consensus_queue.cc:260] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.536130 28734 ts_tablet_manager.cc:1434] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:28.536470 28734 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.538338 28734 tablet_bootstrap.cc:654] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.540548 28736 tablet_bootstrap.cc:654] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.543030 28735 ts_tablet_manager.cc:1929] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.543265 28729 ts_tablet_manager.cc:1929] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.543413 28729 log.cc:1199] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/d547c032332d4d109222b4743fa28561
I20260320 15:27:28.543964 28729 ts_tablet_manager.cc:1950] T d547c032332d4d109222b4743fa28561 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.544857 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 306) took 3015 ms. Trace:
I20260320 15:27:28.544998 28729 rpcz_store.cc:276] 0320 15:27:25.529376 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.530141 (+ 765us) service_pool.cc:225] Handling call
0320 15:27:28.371975 (+2841834us) ts_tablet_manager.cc:1151] Deleting tablet d547c032332d4d109222b4743fa28561
0320 15:27:28.371994 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.437493 (+ 65499us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.524276 (+ 86783us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.544816 (+ 20540us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2819459}
I20260320 15:27:28.545667 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 307) took 2972 ms. Trace:
I20260320 15:27:28.545795 28729 rpcz_store.cc:276] 0320 15:27:25.573213 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.575248 (+ 2035us) service_pool.cc:225] Handling call
0320 15:27:28.545460 (+2970212us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:28.545471 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.545621 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2939207}
I20260320 15:27:28.546274 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 314) took 2622 ms. Trace:
I20260320 15:27:28.546408 28729 rpcz_store.cc:276] 0320 15:27:25.923938 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.924023 (+ 85us) service_pool.cc:225] Handling call
0320 15:27:28.546170 (+2622147us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:28.546177 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.546248 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2620400}
I20260320 15:27:28.546825 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 313) took 2633 ms. Trace:
I20260320 15:27:28.546947 28729 rpcz_store.cc:276] 0320 15:27:25.913036 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.913178 (+ 142us) service_pool.cc:225] Handling call
0320 15:27:28.546731 (+2633553us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:28.546737 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.546801 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2610496}
I20260320 15:27:28.548336 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 315) took 2607 ms. Trace:
I20260320 15:27:28.548898 28729 rpcz_store.cc:276] 0320 15:27:25.940945 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.941011 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:28.548138 (+2607127us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:28.548146 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.548299 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2606885}
I20260320 15:27:28.549623 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 316) took 2607 ms. Trace:
I20260320 15:27:28.554061 28735 log.cc:1199] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:28.554708 28735 ts_tablet_manager.cc:1950] T 0c2339ff2c474a288876f32b85cecc72 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.555915 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 324) took 2325 ms. Trace:
I20260320 15:27:28.556138 28735 rpcz_store.cc:276] 0320 15:27:26.230651 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.230725 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:28.498493 (+2267768us) ts_tablet_manager.cc:1151] Deleting tablet 0c2339ff2c474a288876f32b85cecc72
0320 15:27:28.498506 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.510100 (+ 11594us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.542876 (+ 32776us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.555839 (+ 12963us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2267475}
I20260320 15:27:28.557075 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 329) took 2285 ms. Trace:
I20260320 15:27:28.557296 28735 rpcz_store.cc:276] 0320 15:27:26.271756 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.271867 (+ 111us) service_pool.cc:225] Handling call
0320 15:27:28.556842 (+2284975us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:28.556857 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.557002 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2281066}
I20260320 15:27:28.558058 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 330) took 2279 ms. Trace:
I20260320 15:27:28.558243 28735 rpcz_store.cc:276] 0320 15:27:26.278593 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.278664 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:28.557865 (+2279201us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:28.557878 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.557995 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2278829}
I20260320 15:27:28.558956 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 331) took 2271 ms. Trace:
I20260320 15:27:28.559115 28735 rpcz_store.cc:276] 0320 15:27:26.287922 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.288104 (+ 182us) service_pool.cc:225] Handling call
0320 15:27:28.558770 (+2270666us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:28.558784 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.558899 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2270039}
I20260320 15:27:28.559687 28729 rpcz_store.cc:276] 0320 15:27:25.942292 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.942348 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:28.549482 (+2607134us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:28.549492 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.549591 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2606614}
I20260320 15:27:28.560154 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 318) took 2614 ms. Trace:
I20260320 15:27:28.560292 28729 rpcz_store.cc:276] 0320 15:27:25.945789 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.945848 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:28.560001 (+2614153us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:28.560013 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.560114 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2613894}
I20260320 15:27:28.560631 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 317) took 2615 ms. Trace:
I20260320 15:27:28.560738 28729 rpcz_store.cc:276] 0320 15:27:25.945618 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:25.949019 (+ 3401us) service_pool.cc:225] Handling call
0320 15:27:28.560508 (+2611489us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:28.560517 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.560595 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2611181}
I20260320 15:27:28.561030 28729 tablet_replica.cc:333] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.561265 28729 raft_consensus.cc:2243] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.561012 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 332) took 2264 ms. Trace:
I20260320 15:27:28.561482 28729 raft_consensus.cc:2272] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.563426 28736 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.563642 28736 ts_tablet_manager.cc:1403] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
I20260320 15:27:28.563717 28733 tablet_bootstrap.cc:492] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.561627 28735 rpcz_store.cc:276] 0320 15:27:26.296473 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.296526 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:28.560389 (+2263863us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:28.560403 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.560861 (+ 458us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2262609}
I20260320 15:27:28.563910 28733 ts_tablet_manager.cc:1403] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20260320 15:27:28.564532 28736 raft_consensus.cc:359] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.564739 28733 raft_consensus.cc:359] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.564961 28733 raft_consensus.cc:385] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.565049 28733 raft_consensus.cc:740] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.565403 28733 consensus_queue.cc:260] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.565975 28733 ts_tablet_manager.cc:1434] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:28.566196 29280 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [term 5 LEADER]: Becoming Leader. State: Replica: ee1caebf640d4ec0bf9b28c4f496ca18, State: Running, Role: LEADER
I20260320 15:27:28.567152 29280 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 547, Committed index: 547, Last appended: 4.547, Last appended by leader: 547, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:28.568852 28729 ts_tablet_manager.cc:1916] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.569947 29285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:28.570452 29285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:28.571226 28733 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.580952 28736 raft_consensus.cc:385] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.581115 28733 tablet_bootstrap.cc:654] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.581264 28736 raft_consensus.cc:740] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.581614 28736 consensus_queue.cc:260] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.582288 28736 ts_tablet_manager.cc:1434] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20260320 15:27:28.582710 28736 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.584594 28736 tablet_bootstrap.cc:654] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.587641 28734 tablet_bootstrap.cc:492] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.587913 28734 ts_tablet_manager.cc:1403] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.002s
I20260320 15:27:28.588917 28734 raft_consensus.cc:359] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.589201 28734 raft_consensus.cc:385] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.589315 28734 raft_consensus.cc:740] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.589645 28734 consensus_queue.cc:260] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.596427 28734 ts_tablet_manager.cc:1434] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260320 15:27:28.601857 28734 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.602581 29245 consensus_queue.cc:1048] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:28.607163 28729 ts_tablet_manager.cc:1929] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.599925 29275 raft_consensus.cc:493] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.565284 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 338) took 1866 ms. Trace:
I20260320 15:27:28.618489 29275 raft_consensus.cc:515] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.619653 29275 leader_election.cc:290] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.621227 28735 rpcz_store.cc:276] 0320 15:27:26.699116 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.699195 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:28.564893 (+1865698us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:28.564967 (+ 74us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.565225 (+ 258us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1865375}
I20260320 15:27:28.621526 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.622071 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 339) took 1914 ms. Trace:
W20260320 15:27:28.622603 28239 leader_election.cc:343] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.622539 28735 rpcz_store.cc:276] 0320 15:27:26.707464 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.707566 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:28.621871 (+1914305us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:28.621884 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.622018 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1914001}
I20260320 15:27:28.624672 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 340) took 1907 ms. Trace:
I20260320 15:27:28.624811 28735 rpcz_store.cc:276] 0320 15:27:26.717447 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.717536 (+ 89us) service_pool.cc:225] Handling call
0320 15:27:28.624440 (+1906904us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:28.624452 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.624601 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1906589}
I20260320 15:27:28.625953 28729 log.cc:1199] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5ada1f644b754fbe8b3790b46ec8cdf4
I20260320 15:27:28.620599 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.626315 28444 raft_consensus.cc:2468] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:28.626196 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 341) took 1903 ms. Trace:
I20260320 15:27:28.626477 28735 rpcz_store.cc:276] 0320 15:27:26.722675 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.722752 (+ 77us) service_pool.cc:225] Handling call
0320 15:27:28.625966 (+1903214us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:28.625978 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.626113 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1902894}
I20260320 15:27:28.627053 28240 leader_election.cc:304] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.627239 28735 tablet_replica.cc:333] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.627446 28735 raft_consensus.cc:2243] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.627693 28735 raft_consensus.cc:2272] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.614548 28734 tablet_bootstrap.cc:654] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.629550 28729 ts_tablet_manager.cc:1950] T 5ada1f644b754fbe8b3790b46ec8cdf4 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.630475 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 321) took 2522 ms. Trace:
I20260320 15:27:28.630887 28729 rpcz_store.cc:276] 0320 15:27:26.108331 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.109681 (+ 1350us) service_pool.cc:225] Handling call
0320 15:27:28.560964 (+2451283us) ts_tablet_manager.cc:1151] Deleting tablet 5ada1f644b754fbe8b3790b46ec8cdf4
0320 15:27:28.560972 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.603662 (+ 42690us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.607027 (+ 3365us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.630406 (+ 23379us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2450607}
I20260320 15:27:28.631738 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 325) took 2397 ms. Trace:
I20260320 15:27:28.632398 28729 rpcz_store.cc:276] 0320 15:27:26.234499 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.234599 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:28.631519 (+2396920us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:28.631549 (+ 30us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.631666 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2395476}
I20260320 15:27:28.633145 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 331) took 2356 ms. Trace:
I20260320 15:27:28.633723 28729 rpcz_store.cc:276] 0320 15:27:26.276491 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.276573 (+ 82us) service_pool.cc:225] Handling call
0320 15:27:28.632958 (+2356385us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:28.632968 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.633066 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2355317}
I20260320 15:27:28.634368 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 332) took 2355 ms. Trace:
I20260320 15:27:28.634827 28729 rpcz_store.cc:276] 0320 15:27:26.279364 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.279415 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:28.634216 (+2354801us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:28.634225 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.634312 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2354449}
I20260320 15:27:28.635478 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 326) took 2396 ms. Trace:
I20260320 15:27:28.636070 28729 rpcz_store.cc:276] 0320 15:27:26.238838 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.238909 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:28.635292 (+2396383us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:28.635300 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.635427 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2350981}
I20260320 15:27:28.636755 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 333) took 2348 ms. Trace:
I20260320 15:27:28.637245 28729 rpcz_store.cc:276] 0320 15:27:26.287938 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.288062 (+ 124us) service_pool.cc:225] Handling call
0320 15:27:28.636587 (+2348525us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:28.636595 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.636683 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2342114}
I20260320 15:27:28.637862 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 334) took 2334 ms. Trace:
I20260320 15:27:28.638345 28729 rpcz_store.cc:276] 0320 15:27:26.303483 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.303633 (+ 150us) service_pool.cc:225] Handling call
0320 15:27:28.637704 (+2334071us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:28.637712 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.637807 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2323661}
I20260320 15:27:28.638955 28729 tablet_replica.cc:333] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.639472 28729 raft_consensus.cc:2243] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.640321 28729 raft_consensus.cc:2272] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.641607 28729 ts_tablet_manager.cc:1916] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.642025 28736 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.642220 28736 ts_tablet_manager.cc:1403] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.002s
I20260320 15:27:28.642674 29246 consensus_queue.cc:1048] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:28.643743 29275 raft_consensus.cc:2804] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.643862 29275 raft_consensus.cc:493] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.644029 29275 raft_consensus.cc:3060] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.644114 28736 raft_consensus.cc:359] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.644331 28736 raft_consensus.cc:385] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.644418 28736 raft_consensus.cc:740] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.644729 28736 consensus_queue.cc:260] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.645275 28736 ts_tablet_manager.cc:1434] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:28.646850 29275 raft_consensus.cc:515] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.647933 29275 leader_election.cc:290] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.648030 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.648451 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:28.649158 28239 leader_election.cc:343] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.649534 28444 raft_consensus.cc:3060] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.652949 28444 raft_consensus.cc:2468] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.653645 28240 leader_election.cc:304] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.654264 29275 raft_consensus.cc:2804] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.654472 29275 raft_consensus.cc:697] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:28.654770 29275 consensus_queue.cc:237] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.656410 28735 ts_tablet_manager.cc:1916] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.665383 28729 ts_tablet_manager.cc:1929] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.665696 28729 log.cc:1199] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/557d8be0e4bf42239b2bb4585e7d0f06
I20260320 15:27:28.667755 28736 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.669600 28736 tablet_bootstrap.cc:654] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.673964 28733 tablet_bootstrap.cc:492] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.674149 28733 ts_tablet_manager.cc:1403] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.103s user 0.012s sys 0.007s
I20260320 15:27:28.675050 28733 raft_consensus.cc:359] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.675254 28733 raft_consensus.cc:385] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.675343 28733 raft_consensus.cc:740] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.675607 28733 consensus_queue.cc:260] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.676225 28733 ts_tablet_manager.cc:1434] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:28.676568 28733 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.676764 28734 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.676905 28734 ts_tablet_manager.cc:1403] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.005s
I20260320 15:27:28.677812 28734 raft_consensus.cc:359] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.678014 28734 raft_consensus.cc:385] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.678107 28734 raft_consensus.cc:740] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.678377 28734 consensus_queue.cc:260] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.678534 28733 tablet_bootstrap.cc:654] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.679010 28734 ts_tablet_manager.cc:1434] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:28.681365 28729 ts_tablet_manager.cc:1950] T 557d8be0e4bf42239b2bb4585e7d0f06 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.684624 28734 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.686338 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 339) took 2342 ms. Trace:
I20260320 15:27:28.686499 28729 rpcz_store.cc:276] 0320 15:27:26.344437 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.344740 (+ 303us) service_pool.cc:225] Handling call
0320 15:27:28.638864 (+2294124us) ts_tablet_manager.cc:1151] Deleting tablet 557d8be0e4bf42239b2bb4585e7d0f06
0320 15:27:28.638872 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.660932 (+ 22060us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.665093 (+ 4161us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.686254 (+ 21161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2276050}
I20260320 15:27:28.689437 29275 raft_consensus.cc:493] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.689640 29275 raft_consensus.cc:515] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.690315 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 340) took 1990 ms. Trace:
I20260320 15:27:28.690387 29275 leader_election.cc:290] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.690455 28729 rpcz_store.cc:276] 0320 15:27:26.700033 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.700101 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:28.690132 (+1990031us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:28.690144 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.690270 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1989749}
I20260320 15:27:28.690884 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 341) took 1982 ms. Trace:
I20260320 15:27:28.690984 28729 rpcz_store.cc:276] 0320 15:27:26.708485 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.708563 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:28.690750 (+1982187us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:28.690760 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.690853 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1981028}
I20260320 15:27:28.691476 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 342) took 1979 ms. Trace:
I20260320 15:27:28.691690 28729 rpcz_store.cc:276] 0320 15:27:26.711879 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.712116 (+ 237us) service_pool.cc:225] Handling call
0320 15:27:28.691256 (+1979140us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:28.691267 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.691438 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1978646}
I20260320 15:27:28.692005 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.692234 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 343) took 1973 ms. Trace:
W20260320 15:27:28.692623 28240 leader_election.cc:343] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.692921 28729 rpcz_store.cc:276] 0320 15:27:26.718950 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.719026 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:28.692041 (+1973015us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:28.692053 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.692185 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1972722}
I20260320 15:27:28.693519 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 347) took 1620 ms. Trace:
I20260320 15:27:28.693655 28729 rpcz_store.cc:276] 0320 15:27:27.073139 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.073206 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:28.693342 (+1620136us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:28.693356 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.693471 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1619851}
I20260320 15:27:28.694092 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 348) took 1620 ms. Trace:
I20260320 15:27:28.697739 28729 rpcz_store.cc:276] 0320 15:27:27.073831 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.073883 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:28.693936 (+1620053us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:28.693949 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.694054 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1619826}
I20260320 15:27:28.698347 28729 tablet_replica.cc:333] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.698681 28729 raft_consensus.cc:2243] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.698966 28729 raft_consensus.cc:2272] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.700132 29290 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260320 15:27:28.706876 28734 tablet_bootstrap.cc:654] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.708055 29237 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.708333 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.708357 29237 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.709091 29237 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:28.709458 28239 leader_election.cc:343] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.709628 28239 leader_election.cc:304] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.710063 29275 raft_consensus.cc:2749] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.710706 28736 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.711041 28736 ts_tablet_manager.cc:1403] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.001s
W20260320 15:27:28.711309 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:28.712517 28736 raft_consensus.cc:359] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.712922 28736 raft_consensus.cc:385] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.713034 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_c424c1cd249645e3bed5b49bdf5f75a1 [id=009b402eeb2f4c2a9bf7d0e94a385ab9]
I20260320 15:27:28.713092 28736 raft_consensus.cc:740] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.713620 28736 consensus_queue.cc:260] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.713774 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]
I20260320 15:27:28.714883 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_294d5eabb8df40cd888a0769290746bb [id=0350c3a65a5e449dbbdb140b2d82d60a]
I20260320 15:27:28.716477 28733 tablet_bootstrap.cc:492] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.716737 28733 ts_tablet_manager.cc:1403] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.016s
I20260320 15:27:28.717720 28733 raft_consensus.cc:359] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.718755 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]
I20260320 15:27:28.719297 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]
W20260320 15:27:28.718091 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:28.719519 28733 raft_consensus.cc:385] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.719635 28733 raft_consensus.cc:740] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.719676 28524 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.719915 28733 consensus_queue.cc:260] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.720938 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_818e1205d4f34148aebe2a4206ca41a5 [id=0998e8ce9720447caf0d5e7fbf145fe7]
I20260320 15:27:28.709492 28729 ts_tablet_manager.cc:1916] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.721418 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 [id=0a1df577ad734c34b6cfe3cba679ed7f]
I20260320 15:27:28.723696 29237 raft_consensus.cc:2749] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.724078 29237 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.724224 29237 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.724946 29237 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:28.729107 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:28.730872 28736 ts_tablet_manager.cc:1434] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260320 15:27:28.721937 28733 ts_tablet_manager.cc:1434] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:28.732592 28733 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.736341 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]
I20260320 15:27:28.737649 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_fbb25a9824fc45c7b85abec85ccb75b8 [id=14aef9640e7d427ba93c8b21d7c332c9]
I20260320 15:27:28.738706 28733 tablet_bootstrap.cc:654] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.740787 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]
I20260320 15:27:28.741482 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]
I20260320 15:27:28.741915 28729 ts_tablet_manager.cc:1929] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.742209 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_12077503f4ba430396cb7392d0e64fa7 [id=18d21a3e06bb4a3f9bb6416cd95047e3]
W20260320 15:27:28.742482 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:28.742673 28524 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.731611 28735 ts_tablet_manager.cc:1929] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.742856 28735 log.cc:1199] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:28.743049 29299 raft_consensus.cc:2749] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.743470 28735 ts_tablet_manager.cc:1950] T 993e2fb1561f4317be85e9b5e5816edc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.744222 28736 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.744527 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a5cb91f5ca204475836edc1cb0ee1035 [id=194ce4979c0d44ce81f03c1764a09c78]
I20260320 15:27:28.744553 28729 log.cc:1199] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2ca38bc990634e21a2c38dd840488f8c
I20260320 15:27:28.745060 28729 ts_tablet_manager.cc:1950] T 2ca38bc990634e21a2c38dd840488f8c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.745659 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]
I20260320 15:27:28.746323 28736 tablet_bootstrap.cc:654] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.746343 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]
I20260320 15:27:28.746912 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_4a66f1fda9e9438aad057009144db449 [id=20d83b20838641f0ae67bcd3653a0ef4]
I20260320 15:27:28.747897 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]
I20260320 15:27:28.748921 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]
I20260320 15:27:28.749400 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]
I20260320 15:27:28.750072 28733 tablet_bootstrap.cc:492] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.750272 28733 ts_tablet_manager.cc:1403] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.003s
I20260320 15:27:28.750484 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 349) took 1673 ms. Trace:
I20260320 15:27:28.750604 28729 rpcz_store.cc:276] 0320 15:27:27.077426 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.077483 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:28.698252 (+1620769us) ts_tablet_manager.cc:1151] Deleting tablet 2ca38bc990634e21a2c38dd840488f8c
0320 15:27:28.698267 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.738212 (+ 39945us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.741752 (+ 3540us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.750415 (+ 8663us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1620575}
I20260320 15:27:28.751259 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 350) took 1572 ms. Trace:
I20260320 15:27:28.751230 28733 raft_consensus.cc:359] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.751472 28733 raft_consensus.cc:385] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.751359 28729 rpcz_store.cc:276] 0320 15:27:27.178564 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.178691 (+ 127us) service_pool.cc:225] Handling call
0320 15:27:28.751066 (+1572375us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:28.751078 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.751203 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1559817}
I20260320 15:27:28.751811 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]
I20260320 15:27:28.751605 28733 raft_consensus.cc:740] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.752398 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 [id=2a94c97148eb4bdf8abb574cb9d30204]
I20260320 15:27:28.752305 28733 consensus_queue.cc:260] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.752441 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 351) took 1562 ms. Trace:
I20260320 15:27:28.752550 28729 rpcz_store.cc:276] 0320 15:27:27.190247 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.190372 (+ 125us) service_pool.cc:225] Handling call
0320 15:27:28.752130 (+1561758us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:28.752144 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.752378 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1559695}
I20260320 15:27:28.753161 28733 ts_tablet_manager.cc:1434] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260320 15:27:28.753460 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]
I20260320 15:27:28.753985 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 [id=2d0fe40191c9403ab71159ef1377e21c]
I20260320 15:27:28.755295 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 342) took 1924 ms. Trace:
I20260320 15:27:28.755523 28735 rpcz_store.cc:276] 0320 15:27:26.830765 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.830843 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:28.627118 (+1796275us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:28.627131 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.674663 (+ 47532us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.719336 (+ 44673us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.755237 (+ 35901us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1795905}
I20260320 15:27:28.756124 28735 tablet_replica.cc:333] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.756443 28735 raft_consensus.cc:2243] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.757009 28735 raft_consensus.cc:2272] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.758152 28735 ts_tablet_manager.cc:1916] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.758476 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]
I20260320 15:27:28.759137 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]
I20260320 15:27:28.759684 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]
I20260320 15:27:28.759933 28733 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.760975 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]
I20260320 15:27:28.762531 28736 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.762574 28733 tablet_bootstrap.cc:654] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.762728 28736 ts_tablet_manager.cc:1403] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.001s
I20260320 15:27:28.764212 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_4495058468dd4441be0cbc0668f6bb8b [id=399646419f3b4fd28458144f43bd4553]
I20260320 15:27:28.764752 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2eff9dcb23c04d50b86d3ad60677435c [id=39d38f795d004300a287cdb264a1f8ca]
I20260320 15:27:28.765030 28734 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.765240 28734 ts_tablet_manager.cc:1403] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.081s user 0.001s sys 0.004s
I20260320 15:27:28.765973 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_eb8fe7340adc409c98903887ac581604 [id=3b4245a4113946c2838ad27642634e80]
I20260320 15:27:28.766366 28734 raft_consensus.cc:359] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.766619 28734 raft_consensus.cc:385] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.766773 28734 raft_consensus.cc:740] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.766830 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]
I20260320 15:27:28.767185 28734 consensus_queue.cc:260] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.767954 28734 ts_tablet_manager.cc:1434] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260320 15:27:28.768339 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]
I20260320 15:27:28.768543 28736 raft_consensus.cc:359] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.768792 28736 raft_consensus.cc:385] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.768873 28736 raft_consensus.cc:740] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.769968 28734 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:28.771977 28109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.74.61:44739 timed out after 0.698s (SENT)
I20260320 15:27:28.772099 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 354) took 1572 ms. Trace:
I20260320 15:27:28.772285 28729 rpcz_store.cc:276] 0320 15:27:27.199811 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.199889 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:28.771867 (+1571978us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:28.771878 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.772032 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1564408}
I20260320 15:27:28.772714 28734 tablet_bootstrap.cc:654] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.779844 28736 consensus_queue.cc:260] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.783394 29299 raft_consensus.cc:493] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.783651 29299 raft_consensus.cc:515] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.784550 29299 leader_election.cc:290] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.784965 28736 ts_tablet_manager.cc:1434] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.022s user 0.001s sys 0.002s
I20260320 15:27:28.785481 28736 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.785449 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.785753 28444 raft_consensus.cc:2393] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:28.786743 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1bc86d3d1ff4b3aa8463eb21f619917" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.792526 28524 leader_election.cc:304] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.793041 29299 raft_consensus.cc:3060] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.795611 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_5d4b16da449247bda955fe596e458307 [id=3f666b62287a456ca179d4adffd4dbcb]
I20260320 15:27:28.796145 29299 raft_consensus.cc:2749] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.796432 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]
I20260320 15:27:28.798060 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 355) took 1571 ms. Trace:
I20260320 15:27:28.798297 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 [id=462e3845a6664e418282ec357ae39ef6]
I20260320 15:27:28.798251 28729 rpcz_store.cc:276] 0320 15:27:27.226963 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.227100 (+ 137us) service_pool.cc:225] Handling call
0320 15:27:28.775730 (+1548630us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:28.783364 (+ 7634us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.797972 (+ 14608us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1547548}
I20260320 15:27:28.799124 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 358) took 1566 ms. Trace:
I20260320 15:27:28.799404 28729 rpcz_store.cc:276] 0320 15:27:27.233070 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.233354 (+ 284us) service_pool.cc:225] Handling call
0320 15:27:28.798865 (+1565511us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:28.798885 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.799040 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1565282}
I20260320 15:27:28.800181 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 359) took 1566 ms. Trace:
I20260320 15:27:28.800295 28729 rpcz_store.cc:276] 0320 15:27:27.233168 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.233631 (+ 463us) service_pool.cc:225] Handling call
0320 15:27:28.799978 (+1566347us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:28.799988 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.800124 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1544247}
I20260320 15:27:28.800495 28736 tablet_bootstrap.cc:654] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.801419 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 360) took 1507 ms. Trace:
I20260320 15:27:28.801578 28733 tablet_bootstrap.cc:492] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.801607 28729 rpcz_store.cc:276] 0320 15:27:27.293893 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.294410 (+ 517us) service_pool.cc:225] Handling call
0320 15:27:28.800748 (+1506338us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:28.800758 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.801357 (+ 599us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1505891}
I20260320 15:27:28.801784 28733 ts_tablet_manager.cc:1403] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.017s
I20260320 15:27:28.802253 28729 tablet_replica.cc:333] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.802385 28734 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.802591 28734 ts_tablet_manager.cc:1403] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.003s
I20260320 15:27:28.802763 28733 raft_consensus.cc:359] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.803002 28733 raft_consensus.cc:385] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.803088 28733 raft_consensus.cc:740] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.802625 28729 raft_consensus.cc:2243] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.803459 28734 raft_consensus.cc:359] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.803635 28729 raft_consensus.cc:2272] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.803684 28734 raft_consensus.cc:385] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.803911 28734 raft_consensus.cc:740] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.803896 28733 consensus_queue.cc:260] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.804495 28733 ts_tablet_manager.cc:1434] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:28.804134 28734 consensus_queue.cc:260] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.805089 28735 ts_tablet_manager.cc:1929] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.805281 28734 ts_tablet_manager.cc:1434] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:28.805361 28735 log.cc:1199] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:28.805322 28729 ts_tablet_manager.cc:1916] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.805807 28734 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.805860 28735 ts_tablet_manager.cc:1950] T 093ae84740d5468188b4a56fa16ded9c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.806802 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 343) took 1847 ms. Trace:
I20260320 15:27:28.806937 28735 rpcz_store.cc:276] 0320 15:27:26.959439 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:26.960116 (+ 677us) service_pool.cc:225] Handling call
0320 15:27:28.756044 (+1795928us) ts_tablet_manager.cc:1151] Deleting tablet 093ae84740d5468188b4a56fa16ded9c
0320 15:27:28.756056 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.775560 (+ 19504us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.804941 (+ 29381us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.806727 (+ 1786us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1145216,"tablet-delete.queue_time_us":1795634}
I20260320 15:27:28.807512 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 345) took 1733 ms. Trace:
I20260320 15:27:28.807641 28735 rpcz_store.cc:276] 0320 15:27:27.074344 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.074413 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:28.807301 (+1732888us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:28.807312 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.807430 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1732632}
I20260320 15:27:28.808121 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 346) took 1732 ms. Trace:
I20260320 15:27:28.808269 28734 tablet_bootstrap.cc:654] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.808240 28735 rpcz_store.cc:276] 0320 15:27:27.075458 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.075557 (+ 99us) service_pool.cc:225] Handling call
0320 15:27:28.807957 (+1732400us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:28.807966 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.808068 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1732127}
I20260320 15:27:28.808576 28735 tablet_replica.cc:333] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.811336 28733 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.813099 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]
I20260320 15:27:28.816360 28733 tablet_bootstrap.cc:654] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.819061 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_047ea6293f154736b2bff4875bc8cd43 [id=4685dae9caa842d6bc5fe1c294a37af3]
I20260320 15:27:28.819938 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_3979e77f6aba4d9f80ba813c92287b84 [id=4737d6ecec7041afb0192ab248a093ca]
I20260320 15:27:28.820516 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_1642210236684e85bbcfdf569f9ae5d6 [id=4a69c667d48e4ae0b73e59fd343e4ff4]
I20260320 15:27:28.821054 28734 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.821247 28734 ts_tablet_manager.cc:1403] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260320 15:27:28.822163 28734 raft_consensus.cc:359] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.822410 28734 raft_consensus.cc:385] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.822487 28734 raft_consensus.cc:740] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.822721 28734 consensus_queue.cc:260] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.823204 28734 ts_tablet_manager.cc:1434] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:28.823570 28734 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:28.823745 28735 raft_consensus.cc:2243] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.824019 28735 raft_consensus.cc:2272] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.825021 28735 ts_tablet_manager.cc:1916] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.832593 28735 ts_tablet_manager.cc:1929] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.832752 28735 log.cc:1199] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2ca38bc990634e21a2c38dd840488f8c
I20260320 15:27:28.833178 28735 ts_tablet_manager.cc:1950] T 2ca38bc990634e21a2c38dd840488f8c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.833999 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 347) took 1757 ms. Trace:
I20260320 15:27:28.834131 28735 rpcz_store.cc:276] 0320 15:27:27.076834 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.076895 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:28.808475 (+1731580us) ts_tablet_manager.cc:1151] Deleting tablet 2ca38bc990634e21a2c38dd840488f8c
0320 15:27:28.808485 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.829121 (+ 20636us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.832424 (+ 3303us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.833952 (+ 1528us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14905,"tablet-delete.queue_time_us":1731363}
I20260320 15:27:28.834790 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 348) took 1721 ms. Trace:
I20260320 15:27:28.834914 28735 rpcz_store.cc:276] 0320 15:27:27.113103 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.113171 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:28.834605 (+1721434us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:28.834614 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.834754 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1718966}
I20260320 15:27:28.835458 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 349) took 1721 ms. Trace:
I20260320 15:27:28.835600 28735 rpcz_store.cc:276] 0320 15:27:27.114213 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.114277 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:28.835317 (+1721040us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:28.835327 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.835422 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28,"tablet-delete.queue_time_us":1719669}
I20260320 15:27:28.836057 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 352) took 1678 ms. Trace:
I20260320 15:27:28.836167 28735 rpcz_store.cc:276] 0320 15:27:27.157892 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.157942 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:28.835926 (+1677984us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:28.835934 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.836022 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1671672}
I20260320 15:27:28.837107 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]
I20260320 15:27:28.837361 28736 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.837535 28736 ts_tablet_manager.cc:1403] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.002s
I20260320 15:27:28.837816 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_b3b4c26f9c844ac9a86b26c019bd713e [id=514f5d793c564cd9b7fd8d1fc84970f8]
I20260320 15:27:28.838332 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_134b3b5c79e344f2bf710b12615479be [id=5186e4c77f9f4f59814887534793f0f0]
I20260320 15:27:28.838347 28736 raft_consensus.cc:359] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.838573 28736 raft_consensus.cc:385] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.838675 28736 raft_consensus.cc:740] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.838909 28736 consensus_queue.cc:260] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.839843 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]
I20260320 15:27:28.840431 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]
I20260320 15:27:28.840888 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_62a50c277a2f45bd926514ab39b3b7e4 [id=55547a21133840a3a80a02db2c72676d]
I20260320 15:27:28.842818 28736 ts_tablet_manager.cc:1434] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:28.845037 28736 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.847334 28736 tablet_bootstrap.cc:654] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.847782 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 354) took 1685 ms. Trace:
I20260320 15:27:28.847936 28735 rpcz_store.cc:276] 0320 15:27:27.161845 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.161904 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:28.846103 (+1684199us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:28.847573 (+ 1470us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.847739 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1679321}
I20260320 15:27:28.848480 28735 tablet_replica.cc:333] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.848735 28735 raft_consensus.cc:2243] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.848948 28735 raft_consensus.cc:2272] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.849617 28735 ts_tablet_manager.cc:1916] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.852021 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]
I20260320 15:27:28.852581 28733 tablet_bootstrap.cc:492] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.852887 28733 ts_tablet_manager.cc:1403] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.002s
I20260320 15:27:28.853332 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]
I20260320 15:27:28.853910 28733 raft_consensus.cc:359] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.854153 28733 raft_consensus.cc:385] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.854262 28733 raft_consensus.cc:740] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.854633 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_094bdcada4784500a661a56536bb221b [id=5bed4e07a5c04c2f8950f42423d35bbe]
I20260320 15:27:28.854535 28733 consensus_queue.cc:260] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.855206 28733 ts_tablet_manager.cc:1434] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:28.855917 28733 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.856256 28729 ts_tablet_manager.cc:1929] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.856436 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]
I20260320 15:27:28.856516 28729 log.cc:1199] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:28.857066 28729 ts_tablet_manager.cc:1950] T 591ccc1d349e4fd599d5e343f91defe2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.857349 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]
I20260320 15:27:28.858820 28733 tablet_bootstrap.cc:654] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.859462 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_0f2d272a74ac4a8ab851d662826ba9e6 [id=611935c42b0e42c7b8b6fa719c526910]
I20260320 15:27:28.860107 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]
I20260320 15:27:28.860965 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 363) took 1552 ms. Trace:
I20260320 15:27:28.861150 28729 rpcz_store.cc:276] 0320 15:27:27.308963 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.309028 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:28.802159 (+1493131us) ts_tablet_manager.cc:1151] Deleting tablet 591ccc1d349e4fd599d5e343f91defe2
0320 15:27:28.802168 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.826997 (+ 24829us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.851793 (+ 24796us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.860921 (+ 9128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1492570}
I20260320 15:27:28.861790 28729 tablet_replica.cc:333] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.867774 28729 raft_consensus.cc:2243] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.861816 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343]
I20260320 15:27:28.870615 28735 ts_tablet_manager.cc:1929] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.870858 28735 log.cc:1199] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:28.871364 28735 ts_tablet_manager.cc:1950] T b2355eadcae6483bae7ae475d74dd131 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:28.871827 28729 pending_rounds.cc:70] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:28.864023 28734 tablet_bootstrap.cc:654] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.871940 28729 pending_rounds.cc:77] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387754605248512 op_type: NO_OP noop_request { }
I20260320 15:27:28.872081 28729 raft_consensus.cc:2889] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:28.872314 28729 raft_consensus.cc:2272] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.873149 28736 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.873473 28736 ts_tablet_manager.cc:1403] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.006s
I20260320 15:27:28.874528 28736 raft_consensus.cc:359] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.874809 28736 raft_consensus.cc:385] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.874866 29245 raft_consensus.cc:493] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.875087 29245 raft_consensus.cc:515] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.874905 28736 raft_consensus.cc:740] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.875747 28736 consensus_queue.cc:260] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.875991 29245 leader_election.cc:290] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.876652 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd293a68f894cbfa122d53918816c28" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.876905 28297 raft_consensus.cc:2468] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:28.877089 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 353) took 1717 ms. Trace:
I20260320 15:27:28.877202 28735 rpcz_store.cc:276] 0320 15:27:27.160004 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.160062 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:28.848379 (+1688317us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:28.848389 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.864289 (+ 15900us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.870404 (+ 6115us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.877012 (+ 6608us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":192,"tablet-delete.queue_time_us":1681489}
I20260320 15:27:28.877400 28729 ts_tablet_manager.cc:1916] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.877766 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 359) took 1575 ms. Trace:
I20260320 15:27:28.877868 28735 rpcz_store.cc:276] 0320 15:27:27.302329 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.302404 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:28.877535 (+1575131us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:28.877547 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.877667 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1557464}
I20260320 15:27:28.878186 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd293a68f894cbfa122d53918816c28" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.878290 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 358) took 1576 ms. Trace:
I20260320 15:27:28.878418 28583 raft_consensus.cc:2468] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:28.878396 28735 rpcz_store.cc:276] 0320 15:27:27.302144 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.303203 (+ 1059us) service_pool.cc:225] Handling call
0320 15:27:28.878096 (+1574893us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:28.878107 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.878217 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1516473}
I20260320 15:27:28.878831 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 364) took 1515 ms. Trace:
I20260320 15:27:28.878927 28735 rpcz_store.cc:276] 0320 15:27:27.363213 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.365903 (+ 2690us) service_pool.cc:225] Handling call
0320 15:27:28.878681 (+1512778us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:28.878690 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.878775 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1512566}
I20260320 15:27:28.879317 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 366) took 1515 ms. Trace:
I20260320 15:27:28.879412 28735 rpcz_store.cc:276] 0320 15:27:27.363430 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.366668 (+ 3238us) service_pool.cc:225] Handling call
0320 15:27:28.879167 (+1512499us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:28.879174 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.879263 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1512309}
I20260320 15:27:28.879832 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 367) took 1516 ms. Trace:
I20260320 15:27:28.879926 28735 rpcz_store.cc:276] 0320 15:27:27.363595 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.366890 (+ 3295us) service_pool.cc:225] Handling call
0320 15:27:28.879682 (+1512792us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:28.879690 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.879777 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1512643}
I20260320 15:27:28.880342 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 357) took 1578 ms. Trace:
I20260320 15:27:28.880445 28735 rpcz_store.cc:276] 0320 15:27:27.302033 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.302894 (+ 861us) service_pool.cc:225] Handling call
0320 15:27:28.880174 (+1577280us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:28.880182 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.880288 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1510106}
I20260320 15:27:28.880614 28382 leader_election.cc:304] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.880837 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 377) took 1458 ms. Trace:
I20260320 15:27:28.880934 29245 raft_consensus.cc:2804] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.880932 28735 rpcz_store.cc:276] 0320 15:27:27.422194 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.422245 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:28.880700 (+1458455us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:28.880708 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.880783 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1450978}
I20260320 15:27:28.881047 29245 raft_consensus.cc:493] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.881140 29245 raft_consensus.cc:3060] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.881325 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 374) took 1481 ms. Trace:
I20260320 15:27:28.881407 28735 rpcz_store.cc:276] 0320 15:27:27.399763 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.399825 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:28.881180 (+1481355us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:28.881188 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.881264 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1451137}
I20260320 15:27:28.881811 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 380) took 1451 ms. Trace:
I20260320 15:27:28.881901 28735 rpcz_store.cc:276] 0320 15:27:27.438559 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.438658 (+ 99us) service_pool.cc:225] Handling call
0320 15:27:28.881648 (+1442990us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:28.881658 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.881756 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1436280}
I20260320 15:27:28.882292 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 383) took 1385 ms. Trace:
I20260320 15:27:28.882380 28735 rpcz_store.cc:276] 0320 15:27:27.496625 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.497762 (+ 1137us) service_pool.cc:225] Handling call
0320 15:27:28.882121 (+1384359us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:28.882132 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.882218 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1289859}
I20260320 15:27:28.882789 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 384) took 1283 ms. Trace:
I20260320 15:27:28.882882 28735 rpcz_store.cc:276] 0320 15:27:27.599177 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.599579 (+ 402us) service_pool.cc:225] Handling call
0320 15:27:28.882617 (+1283038us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:28.882627 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.882742 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1251826}
I20260320 15:27:28.883301 28735 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57830 (request call id 388) took 1045 ms. Trace:
I20260320 15:27:28.883913 28736 ts_tablet_manager.cc:1434] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.001s sys 0.002s
I20260320 15:27:28.884122 29245 raft_consensus.cc:515] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.884371 28736 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.884708 29299 raft_consensus.cc:493] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.884876 29245 leader_election.cc:290] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.884902 29299 raft_consensus.cc:515] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.885569 29299 leader_election.cc:290] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.886083 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.886391 28444 raft_consensus.cc:2393] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:28.887037 28736 tablet_bootstrap.cc:654] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.887596 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd293a68f894cbfa122d53918816c28" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:28.887807 28583 raft_consensus.cc:3060] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.883401 28735 rpcz_store.cc:276] 0320 15:27:27.838251 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.838327 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:28.883138 (+1044811us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:28.883149 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.883246 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1044512}
I20260320 15:27:28.888804 28735 tablet_replica.cc:333] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:28.889004 28735 raft_consensus.cc:2243] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:28.889313 28735 pending_rounds.cc:70] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:28.889372 28735 pending_rounds.cc:77] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387755755429888 op_type: NO_OP noop_request { }
I20260320 15:27:28.889493 28735 raft_consensus.cc:2889] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:28.889590 28735 raft_consensus.cc:2272] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.892016 28735 ts_tablet_manager.cc:1916] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.896682 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a9ddea49e641da881fba339ddef549" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:28.897351 29299 raft_consensus.cc:493] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.897660 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dd293a68f894cbfa122d53918816c28" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:28.897872 28297 raft_consensus.cc:3060] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.901463 28297 raft_consensus.cc:2468] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:28.904187 28383 leader_election.cc:304] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.904954 29246 raft_consensus.cc:2804] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.905122 29246 raft_consensus.cc:697] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:28.905427 29246 consensus_queue.cc:237] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.908176 29299 raft_consensus.cc:515] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.908576 28735 ts_tablet_manager.cc:1929] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.908731 28735 log.cc:1199] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/66a9ddea49e641da881fba339ddef549
I20260320 15:27:28.908836 29299 leader_election.cc:290] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.909150 28735 ts_tablet_manager.cc:1950] T 66a9ddea49e641da881fba339ddef549 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:28.910439 28524 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: Tablet not RUNNING: STOPPED
I20260320 15:27:28.910597 28524 leader_election.cc:304] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.911043 28729 ts_tablet_manager.cc:1929] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:28.911188 28729 log.cc:1199] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:28.914861 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "993e2fb1561f4317be85e9b5e5816edc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.915277 28444 raft_consensus.cc:2468] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:28.916697 29299 raft_consensus.cc:3060] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.920185 28729 ts_tablet_manager.cc:1950] T 0c2339ff2c474a288876f32b85cecc72 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.923974 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 362) took 1615 ms. Trace:
I20260320 15:27:28.924247 28729 rpcz_store.cc:276] 0320 15:27:27.308790 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.309749 (+ 959us) service_pool.cc:225] Handling call
0320 15:27:28.861714 (+1551965us) ts_tablet_manager.cc:1151] Deleting tablet 0c2339ff2c474a288876f32b85cecc72
0320 15:27:28.861726 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.891661 (+ 29935us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.910863 (+ 19202us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.923917 (+ 13054us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4480,"tablet-delete.queue_time_us":1551666}
I20260320 15:27:28.924916 28729 tablet_replica.cc:333] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.925246 28729 raft_consensus.cc:2243] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.925490 28729 raft_consensus.cc:2272] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.926332 28524 leader_election.cc:304] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.926592 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]
I20260320 15:27:28.927332 29197 raft_consensus.cc:2804] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.927634 29197 raft_consensus.cc:493] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.927773 29197 raft_consensus.cc:3060] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.928241 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 [id=667652afa87d46bc878b41081fc53171]
I20260320 15:27:28.928743 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]
I20260320 15:27:28.929209 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]
I20260320 15:27:28.929888 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_103c6e9e031144a1bc44f351aa73d587 [id=6a055fea11de4e39addc011ed3a3fe4f]
I20260320 15:27:28.930819 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 [id=6ea6362e6d934bfe8cb54c912b4932dc]
I20260320 15:27:28.931214 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a2607c8356f048f1bb3fbe1f240848c2 [id=713c4ac167384536b97796e4ba84e04e]
I20260320 15:27:28.933961 29197 raft_consensus.cc:515] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.934926 29197 leader_election.cc:290] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:28.935173 28524 leader_election.cc:343] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:28.936558 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "993e2fb1561f4317be85e9b5e5816edc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.936817 28444 raft_consensus.cc:3060] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:28.940215 28524 leader_election.cc:343] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:28.942101 28583 raft_consensus.cc:2468] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:28.942840 28444 raft_consensus.cc:2468] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:28.943605 28524 leader_election.cc:304] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:28.944752 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]
I20260320 15:27:28.944883 29197 raft_consensus.cc:2804] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.945050 29197 raft_consensus.cc:697] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:28.945523 29197 consensus_queue.cc:237] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.946210 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]
I20260320 15:27:28.946357 29299 raft_consensus.cc:2749] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.946650 28734 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:28.946825 28734 ts_tablet_manager.cc:1403] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.123s user 0.005s sys 0.024s
I20260320 15:27:28.947016 29275 raft_consensus.cc:493] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.947209 29275 raft_consensus.cc:515] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.947439 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]
I20260320 15:27:28.947889 28734 raft_consensus.cc:359] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.948073 28734 raft_consensus.cc:385] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.948112 29275 leader_election.cc:290] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.948165 28734 raft_consensus.cc:740] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.948278 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_7b794017a8044d3a82313842a3da9995 [id=792a35a32922494599ee94725386a2a6]
I20260320 15:27:28.948390 28734 consensus_queue.cc:260] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.949052 28729 ts_tablet_manager.cc:1916] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.949211 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:28.949504 28444 raft_consensus.cc:2468] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:28.950012 28240 leader_election.cc:304] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.950431 29275 raft_consensus.cc:2804] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.950555 29275 raft_consensus.cc:493] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.950574 29299 raft_consensus.cc:493] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.950652 29275 raft_consensus.cc:3060] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.950721 29299 raft_consensus.cc:515] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.951380 29299 leader_election.cc:290] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:28.952719 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:28.953567 28733 tablet_bootstrap.cc:492] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:28.953567 29275 raft_consensus.cc:515] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.953869 28733 ts_tablet_manager.cc:1403] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.016s
W20260320 15:27:28.954747 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:28.954962 28524 leader_election.cc:304] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:28.954933 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:28.955185 28447 raft_consensus.cc:3060] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.955281 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:28.955791 28239 leader_election.cc:343] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.956559 29275 leader_election.cc:290] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.956637 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:28.957199 28239 leader_election.cc:343] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:28.957547 28734 ts_tablet_manager.cc:1434] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:28.957777 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]
I20260320 15:27:28.959230 28736 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:28.959476 28736 ts_tablet_manager.cc:1403] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.075s user 0.018s sys 0.031s
I20260320 15:27:28.959868 28447 raft_consensus.cc:2468] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:28.960464 28240 leader_election.cc:304] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:28.960943 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c [id=7dc7c07f0ba646fbb6e8ac8b228e6e5d]
I20260320 15:27:28.960975 29275 raft_consensus.cc:2804] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.961395 29275 raft_consensus.cc:697] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:28.961140 28736 raft_consensus.cc:359] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.961916 28736 raft_consensus.cc:385] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.961854 29275 consensus_queue.cc:237] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.963009 28736 raft_consensus.cc:740] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:28.963325 28736 consensus_queue.cc:260] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:28.963968 27944 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-taskc3SESu/build/asan/bin/kudu pid 28020: Timed out: CheckLeaks RPC to 127.27.74.61:44739 timed out after 1.000s (SENT)
I20260320 15:27:28.964179 27944 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskc3SESu/build/asan/bin/kudu with pid 28020
I20260320 15:27:28.966305 28736 ts_tablet_manager.cc:1434] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260320 15:27:28.966806 29246 raft_consensus.cc:493] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.967021 29246 raft_consensus.cc:515] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.967105 28736 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:28.967167 29299 raft_consensus.cc:2749] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:28.967687 29246 leader_election.cc:290] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.967782 28733 raft_consensus.cc:359] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.968000 28733 raft_consensus.cc:385] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:28.968086 28733 raft_consensus.cc:740] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:28.970423 28383 leader_election.cc:343] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:28.970726 28733 consensus_queue.cc:260] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.971153 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_65a8a44c03cd4385a65c2e752446c84c [id=81f681463e614a42b8730cace88a123a]
I20260320 15:27:28.972146 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_326fcace90ef4e0ba553655d7b3930ae [id=840bfa3d362147309aeded0035f1b3c6]
I20260320 15:27:28.973553 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2355eadcae6483bae7ae475d74dd131" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:28.973950 28583 raft_consensus.cc:2468] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:28.974498 28382 leader_election.cc:304] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:28.975004 29246 raft_consensus.cc:2804] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:28.975129 29246 raft_consensus.cc:493] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:28.975127 28736 tablet_bootstrap.cc:654] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:28.975224 29246 raft_consensus.cc:3060] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.975694 28733 ts_tablet_manager.cc:1434] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260320 15:27:28.976039 28733 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:28.977667 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]
I20260320 15:27:28.978159 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f051dee79ff04e4faa3f02332b20d6e3 [id=8e3d6498194f4cf08a8d6b1acca86844]
I20260320 15:27:28.978560 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_186af3c003da48d78f6c983d6f7b63d8 [id=8f5be23c174e487dacf40ad504ead589]
I20260320 15:27:28.978717 29246 raft_consensus.cc:515] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.978940 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]
I20260320 15:27:28.979362 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_089d0cb58bd1495cbe761dae3b90b562 [id=8fe18e7035b544b3bd5b5ef3da3bd504]
I20260320 15:27:28.979391 28729 ts_tablet_manager.cc:1929] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.979607 28729 log.cc:1199] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b1bc86d3d1ff4b3aa8463eb21f619917
I20260320 15:27:28.979812 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]
I20260320 15:27:28.981262 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2355eadcae6483bae7ae475d74dd131" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:28.981478 28585 raft_consensus.cc:3060] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:28.984706 28585 raft_consensus.cc:2468] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:28.986178 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9]
I20260320 15:27:28.988348 28729 ts_tablet_manager.cc:1950] T b1bc86d3d1ff4b3aa8463eb21f619917 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:28.989323 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 361) took 1681 ms. Trace:
I20260320 15:27:28.989473 29197 raft_consensus.cc:493] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:28.989543 28729 rpcz_store.cc:276] 0320 15:27:27.307901 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.313285 (+ 5384us) service_pool.cc:225] Handling call
0320 15:27:28.924819 (+1611534us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:28.924829 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.975892 (+ 51063us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.979252 (+ 3360us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.989260 (+ 10008us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1588849}
I20260320 15:27:28.989976 28729 tablet_replica.cc:333] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:28.989871 29197 raft_consensus.cc:515] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:28.990172 28729 raft_consensus.cc:2243] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:28.990212 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]
I20260320 15:27:28.990361 28729 raft_consensus.cc:2272] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:28.990958 28729 ts_tablet_manager.cc:1916] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:28.992137 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_3d0dcd71707248d49f09c2356ee00f6c [id=98dd29a311a548d3bc64a6948ac8743c]
I20260320 15:27:28.983777 29246 leader_election.cc:290] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:28.985380 28382 leader_election.cc:304] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:28.993288 29245 raft_consensus.cc:2804] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:28.993458 29245 raft_consensus.cc:697] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:28.993762 29245 consensus_queue.cc:237] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:28.995584 28383 leader_election.cc:343] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:28.998240 28729 ts_tablet_manager.cc:1929] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:28.999053 29197 leader_election.cc:290] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:28.999305 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.002316 28733 tablet_bootstrap.cc:654] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:28.995260 28862 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.74.61:44739 timed out after 0.519s (SENT)
I20260320 15:27:29.000339 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b03e7555b9a646b599acb0c7dff90c9a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.003680 28729 log.cc:1199] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:29.004158 28729 ts_tablet_manager.cc:1950] T b03e7555b9a646b599acb0c7dff90c9a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:29.004979 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 368) took 1651 ms. Trace:
I20260320 15:27:29.005098 28729 rpcz_store.cc:276] 0320 15:27:27.353196 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.353665 (+ 469us) service_pool.cc:225] Handling call
0320 15:27:28.989885 (+1636220us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:28.989899 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:28.994504 (+ 4605us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:28.998081 (+ 3577us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:29.004940 (+ 6859us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1603162}
I20260320 15:27:29.005686 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 367) took 1652 ms. Trace:
I20260320 15:27:29.005801 28729 rpcz_store.cc:276] 0320 15:27:27.352819 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.352879 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:29.005446 (+1652567us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:29.005456 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.005650 (+ 194us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1615813}
I20260320 15:27:29.006160 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 372) took 1625 ms. Trace:
I20260320 15:27:29.006290 28729 rpcz_store.cc:276] 0320 15:27:27.380851 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.381402 (+ 551us) service_pool.cc:225] Handling call
0320 15:27:29.006024 (+1624622us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:29.006034 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.006128 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1608944}
W20260320 15:27:29.006372 28524 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: STOPPED
I20260320 15:27:29.006657 28524 leader_election.cc:304] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.006717 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 373) took 1617 ms. Trace:
I20260320 15:27:29.006814 28729 rpcz_store.cc:276] 0320 15:27:27.388920 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.389173 (+ 253us) service_pool.cc:225] Handling call
0320 15:27:29.006542 (+1617369us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:29.006550 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.006671 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1605669}
I20260320 15:27:29.007113 28729 tablet_replica.cc:333] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:29.008590 28134 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Refusing update from remote peer ee1caebf640d4ec0bf9b28c4f496ca18: Log matching property violated. Preceding OpId in replica: term: 4 index: 537. Preceding OpId from leader: term: 5 index: 548. (index mismatch)
I20260320 15:27:29.009325 29285 consensus_queue.cc:1048] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 548, Last known committed idx: 536, Time since last communication: 0.000s
I20260320 15:27:29.010210 28729 raft_consensus.cc:2243] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:29.010432 28729 raft_consensus.cc:2272] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:29.011144 28729 ts_tablet_manager.cc:1916] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:29.016125 29197 raft_consensus.cc:2749] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.002568 28734 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.018096 28729 ts_tablet_manager.cc:1929] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:29.018411 28729 log.cc:1199] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/bcec6175c1ca49f5b682a29d5c1e235a
I20260320 15:27:29.018965 28729 ts_tablet_manager.cc:1950] T bcec6175c1ca49f5b682a29d5c1e235a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:29.020021 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 369) took 1666 ms. Trace:
I20260320 15:27:29.020246 28729 rpcz_store.cc:276] 0320 15:27:27.353287 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.353686 (+ 399us) service_pool.cc:225] Handling call
0320 15:27:29.007017 (+1653331us) ts_tablet_manager.cc:1151] Deleting tablet bcec6175c1ca49f5b682a29d5c1e235a
0320 15:27:29.007026 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.014670 (+ 7644us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:29.017908 (+ 3238us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:29.019940 (+ 2032us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1604980}
I20260320 15:27:29.020591 28734 tablet_bootstrap.cc:654] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.020956 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 383) took 1577 ms. Trace:
I20260320 15:27:29.021127 28729 rpcz_store.cc:276] 0320 15:27:27.442994 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.443686 (+ 692us) service_pool.cc:225] Handling call
0320 15:27:29.020744 (+1577058us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:29.020756 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.020890 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1574764}
I20260320 15:27:29.021667 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 380) took 1579 ms. Trace:
I20260320 15:27:29.021848 28729 rpcz_store.cc:276] 0320 15:27:27.442643 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.443866 (+ 1223us) service_pool.cc:225] Handling call
0320 15:27:29.021500 (+1577634us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:29.021511 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.021596 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1540234}
I20260320 15:27:29.022207 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_ce843e88d7f24053b0be457f201e11ae [id=9941875dffce49d680808963e123cdbd]
I20260320 15:27:29.022429 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 386) took 1579 ms. Trace:
I20260320 15:27:29.022688 28729 rpcz_store.cc:276] 0320 15:27:27.443387 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.443747 (+ 360us) service_pool.cc:225] Handling call
0320 15:27:29.022220 (+1578473us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:29.022232 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.022360 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1522630}
I20260320 15:27:29.023237 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 389) took 1530 ms. Trace:
I20260320 15:27:29.023391 28729 rpcz_store.cc:276] 0320 15:27:27.492537 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.492591 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:29.023069 (+1530478us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:29.023078 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.023185 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1489103}
I20260320 15:27:29.023474 28733 tablet_bootstrap.cc:492] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.023850 28733 ts_tablet_manager.cc:1403] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.048s user 0.005s sys 0.000s
I20260320 15:27:29.024596 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]
I20260320 15:27:29.024847 28733 raft_consensus.cc:359] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.025089 28733 raft_consensus.cc:385] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.025182 28733 raft_consensus.cc:740] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.025203 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 390) took 1447 ms. Trace:
I20260320 15:27:29.025307 28729 rpcz_store.cc:276] 0320 15:27:27.578201 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.578510 (+ 309us) service_pool.cc:225] Handling call
0320 15:27:29.023737 (+1445227us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:29.024090 (+ 353us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.025138 (+ 1048us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1440702}
I20260320 15:27:29.025817 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 393) took 1149 ms. Trace:
I20260320 15:27:29.025591 28733 consensus_queue.cc:260] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.026041 28729 rpcz_store.cc:276] 0320 15:27:27.876274 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.876336 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:29.025664 (+1149328us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:29.025674 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.025762 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1129666}
I20260320 15:27:29.026525 28729 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39582 (request call id 394) took 1051 ms. Trace:
I20260320 15:27:29.026576 28733 ts_tablet_manager.cc:1434] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:29.026703 28729 rpcz_store.cc:276] 0320 15:27:27.992098 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:27.992716 (+ 618us) service_pool.cc:225] Handling call
0320 15:27:29.026352 (+1033636us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:29.026359 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:29.026461 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":966213}
I20260320 15:27:29.027285 28729 tablet_replica.cc:333] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:29.027598 28729 raft_consensus.cc:2243] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:29.027882 28729 raft_consensus.cc:2272] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:29.027957 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_fda3998f623540e6beca0a6f3455b0b3 [id=9e38178c12314306a0b595904a402f21]
I20260320 15:27:29.028584 28729 ts_tablet_manager.cc:1916] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:29.030380 28733 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.031381 28734 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.031595 28734 ts_tablet_manager.cc:1403] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.073s user 0.003s sys 0.000s
I20260320 15:27:29.032634 28734 raft_consensus.cc:359] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.032900 28734 raft_consensus.cc:385] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.032982 28734 raft_consensus.cc:740] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.033336 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]
I20260320 15:27:29.033571 28734 consensus_queue.cc:260] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.034847 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]
I20260320 15:27:29.036285 28729 ts_tablet_manager.cc:1929] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:29.036435 28729 log.cc:1199] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/66a9ddea49e641da881fba339ddef549
I20260320 15:27:29.036899 28729 ts_tablet_manager.cc:1950] T 66a9ddea49e641da881fba339ddef549 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:29.037936 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]
I20260320 15:27:29.038259 28734 ts_tablet_manager.cc:1434] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260320 15:27:29.038779 28736 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.038832 28733 tablet_bootstrap.cc:654] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.038988 28736 ts_tablet_manager.cc:1403] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.072s user 0.003s sys 0.008s
I20260320 15:27:29.039664 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_fb580f5b6e1141e1a236805952cf1214 [id=a497706a56ab423cb61065e803c64897]
I20260320 15:27:29.040103 28736 raft_consensus.cc:359] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.040369 28736 raft_consensus.cc:385] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.040457 28736 raft_consensus.cc:740] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.040513 28734 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.040776 28736 consensus_queue.cc:260] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.041361 28736 ts_tablet_manager.cc:1434] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:29.042488 28736 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.043494 28734 tablet_bootstrap.cc:654] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.044703 28736 tablet_bootstrap.cc:654] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.047132 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 [id=a6da90b5243e4ddc93342b80dac5e784]
I20260320 15:27:29.050586 29197 raft_consensus.cc:493] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.050808 29197 raft_consensus.cc:515] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.051635 29197 leader_election.cc:290] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:29.052664 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.053515 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931f6f88a1d94a40b69f07216b8313c8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.053826 28447 raft_consensus.cc:2393] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:29.054332 28524 leader_election.cc:304] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.054555 29245 raft_consensus.cc:493] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.054757 29245 raft_consensus.cc:515] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.054970 29197 raft_consensus.cc:3060] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.055647 29245 leader_election.cc:290] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.056262 28736 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.056420 28736 ts_tablet_manager.cc:1403] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.007s
I20260320 15:27:29.056917 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cfad5cf843a437e8e15c7236d1dc5fe" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.057294 28736 raft_consensus.cc:359] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.057463 28736 raft_consensus.cc:385] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.057550 28736 raft_consensus.cc:740] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.057816 28736 consensus_queue.cc:260] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:29.058261 28382 leader_election.cc:343] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.058456 28736 ts_tablet_manager.cc:1434] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:29.058679 29197 raft_consensus.cc:2749] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.060039 28733 tablet_bootstrap.cc:492] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.060289 28733 ts_tablet_manager.cc:1403] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.001s
I20260320 15:27:29.060434 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]
I20260320 15:27:29.061322 28733 raft_consensus.cc:359] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.061854 28733 raft_consensus.cc:385] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.061976 28733 raft_consensus.cc:740] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.062340 28733 consensus_queue.cc:260] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.066597 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a74831451d0047c2a1a67b8b8e2b93d0 [id=ada35f69f6904e7b8d50889e605f0070]
I20260320 15:27:29.067915 28736 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.067967 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_dfd551c35b134e27a2424da18f1a5bd3 [id=b2921bb259f24e1682ec25e2f083ff5f]
I20260320 15:27:29.068795 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cfad5cf843a437e8e15c7236d1dc5fe" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.069083 28297 raft_consensus.cc:2468] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.069397 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]
I20260320 15:27:29.069761 28383 leader_election.cc:304] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.069928 28736 tablet_bootstrap.cc:654] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.070745 29245 raft_consensus.cc:2804] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.070863 29245 raft_consensus.cc:493] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.070984 29245 raft_consensus.cc:3060] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.074265 29245 raft_consensus.cc:515] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.076280 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cfad5cf843a437e8e15c7236d1dc5fe" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.076532 28297 raft_consensus.cc:3060] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.076702 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cfad5cf843a437e8e15c7236d1dc5fe" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:29.077528 28382 leader_election.cc:343] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.077998 29245 leader_election.cc:290] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.078408 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]
I20260320 15:27:29.078987 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]
I20260320 15:27:29.079602 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]
I20260320 15:27:29.079674 28734 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.080008 28734 ts_tablet_manager.cc:1403] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.040s user 0.018s sys 0.005s
I20260320 15:27:29.080210 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]
I20260320 15:27:29.079458 28297 raft_consensus.cc:2468] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.080801 28733 ts_tablet_manager.cc:1434] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260320 15:27:29.080917 28383 leader_election.cc:304] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.081478 29245 raft_consensus.cc:2804] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.082051 29245 raft_consensus.cc:697] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.082445 29245 consensus_queue.cc:237] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.086756 28733 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.085721 28734 raft_consensus.cc:359] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.087935 28734 raft_consensus.cc:385] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.088577 28734 raft_consensus.cc:740] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.088685 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_42657ecb01cc40af923263a4556db5df [id=b9bf566229c34b7e84767938e2f605ff]
I20260320 15:27:29.089296 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]
I20260320 15:27:29.089579 28733 tablet_bootstrap.cc:654] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.089258 28734 consensus_queue.cc:260] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.089831 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed]
I20260320 15:27:29.090358 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]
I20260320 15:27:29.090837 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_36123e5c9fed4e51bfab33f47ed1d82b [id=bf25edfb9f0e4924aadf3e85e57fab77]
I20260320 15:27:29.090900 28734 ts_tablet_manager.cc:1434] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260320 15:27:29.092072 28734 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.092221 28736 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.092545 28736 ts_tablet_manager.cc:1403] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.004s
I20260320 15:27:29.093694 28736 raft_consensus.cc:359] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.094000 28736 raft_consensus.cc:385] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.094092 28736 raft_consensus.cc:740] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.094529 28736 consensus_queue.cc:260] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.096730 28736 ts_tablet_manager.cc:1434] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:29.097131 28736 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.099037 28736 tablet_bootstrap.cc:654] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.100598 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_5b5bbe0346774e8cbb24c184595d322e [id=c5d8e4f7585a4d72af6d2af8cff2950e]
I20260320 15:27:29.101542 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d643a0220e4847efb0a5e33127808a91 [id=c759d0cd0eb64c76812102f626211d6d]
I20260320 15:27:29.102030 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_c19d1d44b172435dbb61e970e1b3ced2 [id=c825e1c06ebc423c98aadd08b90bfdc0]
I20260320 15:27:29.103044 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]
I20260320 15:27:29.105324 28734 tablet_bootstrap.cc:654] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.106204 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]
I20260320 15:27:29.108709 28733 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.108911 28733 ts_tablet_manager.cc:1403] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.002s
I20260320 15:27:29.109862 28733 raft_consensus.cc:359] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.110095 28733 raft_consensus.cc:385] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.110208 28733 raft_consensus.cc:740] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.110469 28733 consensus_queue.cc:260] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.111181 28733 ts_tablet_manager.cc:1434] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:29.112684 28734 tablet_bootstrap.cc:492] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.113469 28734 ts_tablet_manager.cc:1403] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20260320 15:27:29.114437 28734 raft_consensus.cc:359] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.114681 28734 raft_consensus.cc:385] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.114768 28734 raft_consensus.cc:740] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.115069 28734 consensus_queue.cc:260] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.115934 28733 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.116171 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_602a7434dd7d43a798b06bff668c8d0e [id=cefb97d71bd946b3b75e6d122314bc93]
I20260320 15:27:29.116670 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_ef4a998b01e2437a8edcf95a89e5a314 [id=cf3dff3da9334df6a176702917cfaf18]
I20260320 15:27:29.117247 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]
I20260320 15:27:29.118001 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_c2a40da69e9d4735b3dd5e71d419b00b [id=cfa06a2552dd47ca947741764a0bf762]
I20260320 15:27:29.118095 28733 tablet_bootstrap.cc:654] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.119338 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]
I20260320 15:27:29.119494 29197 raft_consensus.cc:493] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.119721 29197 raft_consensus.cc:515] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.120182 29320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:29.120534 29320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:29.120569 29197 leader_election.cc:290] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:29.121069 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]
I20260320 15:27:29.121639 29197 raft_consensus.cc:493] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.121861 29197 raft_consensus.cc:515] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.121718 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]
I20260320 15:27:29.122591 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_e138abbe7dcc46d1babaa051f60268e7 [id=d84ac38833fe43cfbaefb2f47f7a3af4]
I20260320 15:27:29.122676 29197 leader_election.cc:290] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:29.123317 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c752fb99ef494bda9d144d0e180e3100" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.123659 28297 raft_consensus.cc:2468] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:29.124785 28734 ts_tablet_manager.cc:1434] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:29.125221 28734 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.125674 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c752fb99ef494bda9d144d0e180e3100" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.125910 28447 raft_consensus.cc:2468] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:29.126192 29285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:29.126437 29285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:29.129489 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]
I20260320 15:27:29.132175 28524 leader_election.cc:304] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:29.132413 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_1bbba63ee33c407f8d02e8f63df7245e [id=e703fc2e6e2345138d5a6852097ade20]
I20260320 15:27:29.132683 29197 raft_consensus.cc:2804] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.132794 29197 raft_consensus.cc:493] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.132902 29197 raft_consensus.cc:3060] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.133694 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_4068b4f067d74effb3bc0dc5bb1faddc [id=efaa41b4075b4bc98bf0b19fe0c05ba9]
I20260320 15:27:29.134128 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]
I20260320 15:27:29.134819 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_2827438ad3b74f42968ceb52cf633bff [id=f1748199619b4638917812d77feed8f3]
I20260320 15:27:29.135265 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_536e57665fa14e0e915512c1a27ab8fe [id=f2497b3b3f73471e872d6cba3e653118]
I20260320 15:27:29.135754 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_28bb9bed388041caa7c69d121e4dbabe [id=f2e791aa9a37498fa24b1904cfc045ea]
I20260320 15:27:29.136229 29197 raft_consensus.cc:515] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.136443 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]
I20260320 15:27:29.137140 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_57819bbc59d24aa1a4d2a28ac5e50728 [id=f74f702b87f24a15bcdca554d3a0bbdd]
I20260320 15:27:29.137673 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]
I20260320 15:27:29.138197 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]
I20260320 15:27:29.138715 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]
I20260320 15:27:29.138739 29197 leader_election.cc:290] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:29.139089 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c752fb99ef494bda9d144d0e180e3100" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.139216 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_19d7fded966f4dfcad98437bdbc0e75c [id=fc0da964b571404bbd36fc2257c95c06]
I20260320 15:27:29.139328 28444 raft_consensus.cc:3060] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.141907 29320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:29.142241 29320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [sys.catalog]: This master's current role is: FOLLOWER
W20260320 15:27:29.142874 28524 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b8f3ee3c611d41b483bd35f868042d05
W20260320 15:27:29.143414 28524 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:29.147276 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c752fb99ef494bda9d144d0e180e3100" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.147639 28297 raft_consensus.cc:3060] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.148958 28734 tablet_bootstrap.cc:654] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:29.149550 28640 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.149856 28862 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.150169 28862 leader_election.cc:336] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.150410 28640 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.27.74.61:44739)
W20260320 15:27:29.150528 28862 leader_election.cc:336] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.150676 28640 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.27.74.61:44739)
W20260320 15:27:29.151129 28640 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.27.74.61:44739)
W20260320 15:27:29.151834 28640 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.74.61:44739)
W20260320 15:27:29.151816 28381 connection.cc:537] server connection from 127.0.0.1:39582 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.152340 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]
I20260320 15:27:29.152882 29290 catalog_manager.cc:679] Loaded metadata for table default.Table_d4436c4c1a2a4ad1af830b4d62491073 [id=fe749d518f7b425c91fa96254362fb04]
W20260320 15:27:29.157003 28237 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.157370 29156 raft_consensus.cc:493] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.157593 29156 raft_consensus.cc:515] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.158380 29156 leader_election.cc:290] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:29.158390 28240 connection.cc:537] server connection from 127.0.0.1:57830 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.158771 28380 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.162052 28109 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:29.162367 29340 negotiation.cc:337] Failed RPC negotiation. Trace:
0320 15:27:29.153231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.74.61:44739 (local address 127.0.0.1:50374)
0320 15:27:29.153972 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0320 15:27:29.153999 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0320 15:27:29.154366 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0320 15:27:29.160364 (+ 5998us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":1569,"threads_started":1}
W20260320 15:27:29.162447 28109 leader_election.cc:336] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.162794 28109 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
W20260320 15:27:29.163599 28109 connection.cc:537] server connection from 127.0.0.1:49412 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.165088 28444 raft_consensus.cc:2468] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:29.172973 29290 tablet_loader.cc:96] loaded metadata for tablet 008621b6a3fc40149a1c6aff8b12ef4e (table default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 [id=462e3845a6664e418282ec357ae39ef6])
I20260320 15:27:29.173878 29290 tablet_loader.cc:96] loaded metadata for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad])
I20260320 15:27:29.174528 29290 tablet_loader.cc:96] loaded metadata for tablet 01078353a3a343a3a1231245a391cf8e (table default.Table_42657ecb01cc40af923263a4556db5df [id=b9bf566229c34b7e84767938e2f605ff])
I20260320 15:27:29.175571 29290 tablet_loader.cc:96] loaded metadata for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e])
I20260320 15:27:29.176368 29290 tablet_loader.cc:96] loaded metadata for tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953])
I20260320 15:27:29.177949 29290 tablet_loader.cc:96] loaded metadata for tablet 0acad11980f842e7b3a09216241e493d (table default.Table_1bbba63ee33c407f8d02e8f63df7245e [id=e703fc2e6e2345138d5a6852097ade20])
I20260320 15:27:29.178912 29290 tablet_loader.cc:96] loaded metadata for tablet 0d21b0e8db1a424ea329b8f9cd03ab5b (table default.Table_4495058468dd4441be0cbc0668f6bb8b [id=399646419f3b4fd28458144f43bd4553])
I20260320 15:27:29.178910 29275 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.179217 29275 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.179426 29290 tablet_loader.cc:96] loaded metadata for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b])
I20260320 15:27:29.180231 29275 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.180405 29290 tablet_loader.cc:96] loaded metadata for tablet 11bef661385b4221a991710ffb34dccb (table default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 [id=2a94c97148eb4bdf8abb574cb9d30204])
I20260320 15:27:29.181623 29290 tablet_loader.cc:96] loaded metadata for tablet 197f1935099140a98c5eb801cf44046a (table default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6])
I20260320 15:27:29.182323 29290 tablet_loader.cc:96] loaded metadata for tablet 1bd3032950254e6f9db8da35dbb1acc5 (table default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653])
I20260320 15:27:29.183400 29290 tablet_loader.cc:96] loaded metadata for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589])
I20260320 15:27:29.183341 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.183660 28300 raft_consensus.cc:2468] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.184875 29290 tablet_loader.cc:96] loaded metadata for tablet 20eec704800e4d8b924d97556cc2700e (table default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868])
I20260320 15:27:29.186053 29290 tablet_loader.cc:96] loaded metadata for tablet 233f6a8e9369411b8e0c35a167c55ff6 (table default.Table_7b794017a8044d3a82313842a3da9995 [id=792a35a32922494599ee94725386a2a6])
I20260320 15:27:29.186570 29290 tablet_loader.cc:96] loaded metadata for tablet 23eda54577404cc6900cb3ae348604d4 (table default.Table_36123e5c9fed4e51bfab33f47ed1d82b [id=bf25edfb9f0e4924aadf3e85e57fab77])
I20260320 15:27:29.187233 29290 tablet_loader.cc:96] loaded metadata for tablet 2548d7b8d31049519a85490772621352 (table default.Table_a6c99d8d7f7e4ac0a0785c700d9e4096 [id=ba466e4f2691499c801ab1e748f61aed])
I20260320 15:27:29.188082 29290 tablet_loader.cc:96] loaded metadata for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_ce843e88d7f24053b0be457f201e11ae [id=9941875dffce49d680808963e123cdbd])
I20260320 15:27:29.188683 29290 tablet_loader.cc:96] loaded metadata for tablet 27ccc693e7704650bd77e0c4ca11862e (table default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1])
I20260320 15:27:29.166043 28734 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.188999 28734 ts_tablet_manager.cc:1403] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.003s
I20260320 15:27:29.189536 29320 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260320 15:27:29.190009 28734 raft_consensus.cc:359] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.190224 28734 raft_consensus.cc:385] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.190373 28734 raft_consensus.cc:740] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.190675 28734 consensus_queue.cc:260] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.191454 28734 ts_tablet_manager.cc:1434] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260320 15:27:29.155387 28521 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.159330 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.192133 28524 leader_election.cc:304] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.192449 28583 raft_consensus.cc:2468] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:29.192682 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 49267.4us
I20260320 15:27:29.193068 28382 leader_election.cc:304] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.193276 28524 leader_election.cc:304] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.193974 28734 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.193640 29197 raft_consensus.cc:2749] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.196246 29197 raft_consensus.cc:2804] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.196460 29197 raft_consensus.cc:697] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:29.196869 28734 tablet_bootstrap.cc:654] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.196894 29197 consensus_queue.cc:237] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:29.161716 28523 connection.cc:537] server connection from 127.0.0.1:51060 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:29.199091 29290 tablet_loader.cc:96] loaded metadata for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79])
I20260320 15:27:29.199791 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.199898 29290 tablet_loader.cc:96] loaded metadata for tablet 2cb7cbe0a9224412880144927ec8fa99 (table default.Table_a5cb91f5ca204475836edc1cb0ee1035 [id=194ce4979c0d44ce81f03c1764a09c78])
I20260320 15:27:29.200244 29290 tablet_loader.cc:96] loaded metadata for tablet 2e9791ac61b34fc09a8292c79f0c4032 (table default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1])
I20260320 15:27:29.200553 29245 raft_consensus.cc:2804] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.200685 29290 tablet_loader.cc:96] loaded metadata for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340])
I20260320 15:27:29.200697 29245 raft_consensus.cc:493] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.200832 29245 raft_consensus.cc:3060] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.201361 29290 tablet_loader.cc:96] loaded metadata for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc])
W20260320 15:27:29.208942 28239 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.212739 28734 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.212953 28734 ts_tablet_manager.cc:1403] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.007s
I20260320 15:27:29.213908 28734 raft_consensus.cc:359] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.214143 28734 raft_consensus.cc:385] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.214246 28734 raft_consensus.cc:740] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.214530 28734 consensus_queue.cc:260] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.215196 28734 ts_tablet_manager.cc:1434] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:29.215632 28734 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.216639 29245 raft_consensus.cc:515] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.217621 28734 tablet_bootstrap.cc:654] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.218672 29245 leader_election.cc:290] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.218709 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.219069 28300 raft_consensus.cc:3060] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.221709 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2165371f4b5448c7a2625926082ee4c6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.221983 28583 raft_consensus.cc:3060] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.222502 28736 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.222679 28736 ts_tablet_manager.cc:1403] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.126s user 0.003s sys 0.000s
I20260320 15:27:29.222986 29290 tablet_loader.cc:96] loaded metadata for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_62a50c277a2f45bd926514ab39b3b7e4 [id=55547a21133840a3a80a02db2c72676d])
I20260320 15:27:29.223568 29290 tablet_loader.cc:96] loaded metadata for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_5b5bbe0346774e8cbb24c184595d322e [id=c5d8e4f7585a4d72af6d2af8cff2950e])
I20260320 15:27:29.223702 28736 raft_consensus.cc:359] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.223913 28736 raft_consensus.cc:385] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.224016 28736 raft_consensus.cc:740] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.224260 28736 consensus_queue.cc:260] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.224901 28736 ts_tablet_manager.cc:1434] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:29.225227 28736 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.227931 28736 tablet_bootstrap.cc:654] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.230115 28733 tablet_bootstrap.cc:492] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.230443 28733 ts_tablet_manager.cc:1403] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.115s user 0.023s sys 0.054s
I20260320 15:27:29.231400 28733 raft_consensus.cc:359] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.231745 29245 raft_consensus.cc:493] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.231954 29245 raft_consensus.cc:515] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.232756 29245 leader_election.cc:290] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.232982 28733 raft_consensus.cc:385] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.233955 28583 raft_consensus.cc:2468] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.233997 28733 raft_consensus.cc:740] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.234200 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.234656 28733 consensus_queue.cc:260] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.234939 28585 raft_consensus.cc:2468] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.235288 28733 ts_tablet_manager.cc:1434] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260320 15:27:29.235703 28733 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.236359 28382 leader_election.cc:304] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.236750 28382 leader_election.cc:304] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.237044 29245 raft_consensus.cc:2804] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.237187 29245 raft_consensus.cc:697] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.237598 28733 tablet_bootstrap.cc:654] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.237486 29245 consensus_queue.cc:237] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.237890 28297 raft_consensus.cc:2468] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:29.238729 29246 raft_consensus.cc:2804] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.238865 29246 raft_consensus.cc:493] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.238970 29246 raft_consensus.cc:3060] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.244323 28736 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.244601 28736 ts_tablet_manager.cc:1403] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260320 15:27:29.245540 28736 raft_consensus.cc:359] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.245822 28736 raft_consensus.cc:385] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.245920 28736 raft_consensus.cc:740] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.246253 28736 consensus_queue.cc:260] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.247206 28736 ts_tablet_manager.cc:1434] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.247640 28736 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.252020 28736 tablet_bootstrap.cc:654] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.254499 28733 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.255831 28733 ts_tablet_manager.cc:1403] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260320 15:27:29.258245 28733 raft_consensus.cc:359] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.258662 28733 raft_consensus.cc:385] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.258769 29275 raft_consensus.cc:493] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.258838 28733 raft_consensus.cc:740] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.258944 29275 raft_consensus.cc:515] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.259109 28733 consensus_queue.cc:260] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.259707 28733 ts_tablet_manager.cc:1434] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:29.259749 29275 leader_election.cc:290] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.260038 28733 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.262066 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.262450 28297 raft_consensus.cc:2468] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.262598 28585 raft_consensus.cc:3060] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.263043 28733 tablet_bootstrap.cc:654] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.267663 29275 raft_consensus.cc:493] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.267902 29275 raft_consensus.cc:515] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.268724 29275 leader_election.cc:290] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.269958 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bc3fb47136d4b88a2898af399d74a8a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.270396 29156 raft_consensus.cc:493] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.270601 29156 raft_consensus.cc:515] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:29.272119 28524 consensus_peers.cc:597] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcec6175c1ca49f5b682a29d5c1e235a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.273293 28446 raft_consensus.cc:2468] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.273248 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.275789 28240 leader_election.cc:304] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:29.276244 28240 leader_election.cc:343] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.276585 29156 leader_election.cc:290] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.277233 29246 raft_consensus.cc:515] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.279778 29241 raft_consensus.cc:2804] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.279946 29241 raft_consensus.cc:493] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.280045 29241 raft_consensus.cc:3060] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.280719 28734 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.280867 28734 ts_tablet_manager.cc:1403] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.065s user 0.017s sys 0.029s
I20260320 15:27:29.281730 28734 raft_consensus.cc:359] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.281914 28734 raft_consensus.cc:385] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.281989 28734 raft_consensus.cc:740] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.282196 28734 consensus_queue.cc:260] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.282804 28734 ts_tablet_manager.cc:1434] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:29.283115 28734 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.284235 29241 raft_consensus.cc:515] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.284945 28734 tablet_bootstrap.cc:654] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.286166 28733 tablet_bootstrap.cc:492] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.286383 28733 ts_tablet_manager.cc:1403] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.005s
I20260320 15:27:29.287284 28733 raft_consensus.cc:359] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.287506 28733 raft_consensus.cc:385] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.287611 28733 raft_consensus.cc:740] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.287868 28733 consensus_queue.cc:260] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.288470 28733 ts_tablet_manager.cc:1434] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.288831 28733 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.289511 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.290041 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bc3fb47136d4b88a2898af399d74a8a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.290266 28587 raft_consensus.cc:2468] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.291285 29275 raft_consensus.cc:493] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.293620 29275 raft_consensus.cc:515] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.295099 29275 leader_election.cc:290] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:29.292968 28240 consensus_peers.cc:597] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 591ccc1d349e4fd599d5e343f91defe2. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.299067 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.299417 28447 raft_consensus.cc:2468] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.300841 28240 leader_election.cc:304] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.301859 28734 tablet_bootstrap.cc:492] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.302084 28734 ts_tablet_manager.cc:1403] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.011s
I20260320 15:27:29.302923 29241 leader_election.cc:290] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.302983 28734 raft_consensus.cc:359] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.303223 28734 raft_consensus.cc:385] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.303311 28734 raft_consensus.cc:740] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.303455 29241 raft_consensus.cc:2804] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.303659 29241 raft_consensus.cc:493] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.303637 28734 consensus_queue.cc:260] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.303917 29241 raft_consensus.cc:3060] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.304322 28734 ts_tablet_manager.cc:1434] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:29.305147 29246 leader_election.cc:290] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.305876 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.306071 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.306316 28587 raft_consensus.cc:3060] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.306638 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a072859f39c4c28ae65b0e9afa79d66" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:29.306748 28239 leader_election.cc:343] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.306849 28298 raft_consensus.cc:3060] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.306915 28239 leader_election.cc:304] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.310057 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.310209 28298 raft_consensus.cc:2468] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.310554 29356 consensus_queue.cc:1048] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:29.310930 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.252107 28300 raft_consensus.cc:2468] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:29.311674 28382 leader_election.cc:343] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:29.312493 28862 consensus_peers.cc:597] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 -> Peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Couldn't send request to peer 07e8bb8819eb4cb4892e0cba1f758892. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:29.312649 28862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 70502us
I20260320 15:27:29.313120 29290 tablet_loader.cc:96] loaded metadata for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_047ea6293f154736b2bff4875bc8cd43 [id=4685dae9caa842d6bc5fe1c294a37af3])
I20260320 15:27:29.313356 29356 raft_consensus.cc:2749] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.313462 29275 raft_consensus.cc:493] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.313699 29275 raft_consensus.cc:515] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.314123 28383 leader_election.cc:304] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.314496 29275 leader_election.cc:290] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.315835 29245 raft_consensus.cc:2804] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.316020 29245 raft_consensus.cc:697] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.316075 28733 tablet_bootstrap.cc:654] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.316233 28734 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.316311 29245 consensus_queue.cc:237] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.317981 28734 tablet_bootstrap.cc:654] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.320075 28736 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.320261 28736 ts_tablet_manager.cc:1403] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.073s user 0.019s sys 0.004s
I20260320 15:27:29.321085 28736 raft_consensus.cc:359] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.321295 28736 raft_consensus.cc:385] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.321420 28736 raft_consensus.cc:740] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.321774 28736 consensus_queue.cc:260] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.322396 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.322543 28736 ts_tablet_manager.cc:1434] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:29.322877 28736 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.323014 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bc3fb47136d4b88a2898af399d74a8a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.323174 28585 raft_consensus.cc:3060] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.324894 28736 tablet_bootstrap.cc:654] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.326284 28585 raft_consensus.cc:2468] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.327225 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7b53bb7444a39b471218c6039542f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.327487 28583 raft_consensus.cc:2468] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.334488 28587 raft_consensus.cc:2468] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.335075 28736 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.335896 28736 ts_tablet_manager.cc:1403] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.007s
W20260320 15:27:29.337035 28240 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.337206 28240 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.337553 29357 raft_consensus.cc:2749] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.339212 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bc3fb47136d4b88a2898af399d74a8a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.339557 28444 raft_consensus.cc:3060] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.337217 28736 raft_consensus.cc:359] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.339803 28736 raft_consensus.cc:385] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.339880 28736 raft_consensus.cc:740] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.340607 28736 consensus_queue.cc:260] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.339214 28734 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.341732 28734 ts_tablet_manager.cc:1403] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.012s
I20260320 15:27:29.342638 28734 raft_consensus.cc:359] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.342824 28734 raft_consensus.cc:385] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.342854 28736 ts_tablet_manager.cc:1434] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260320 15:27:29.342924 28734 raft_consensus.cc:740] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.343119 28444 raft_consensus.cc:2468] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.343250 28734 consensus_queue.cc:260] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.343829 28734 ts_tablet_manager.cc:1434] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:29.344192 28734 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.344892 28736 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.345862 28734 tablet_bootstrap.cc:654] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.346750 29357 raft_consensus.cc:493] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.347023 29357 raft_consensus.cc:515] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.347848 29357 leader_election.cc:290] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.348781 28240 leader_election.cc:304] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.349138 29357 raft_consensus.cc:2804] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.349287 29357 raft_consensus.cc:697] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.349587 29357 consensus_queue.cc:237] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.351344 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7b53bb7444a39b471218c6039542f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.351701 28444 raft_consensus.cc:2468] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.352334 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82c47ae3786401d931bb0b486ca3570" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.352624 28447 raft_consensus.cc:2468] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.352707 28736 tablet_bootstrap.cc:654] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.353948 28240 leader_election.cc:304] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.354441 29357 raft_consensus.cc:2804] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.354449 29356 raft_consensus.cc:493] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.354588 29357 raft_consensus.cc:493] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.354687 29357 raft_consensus.cc:3060] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.354620 29356 raft_consensus.cc:515] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.355557 29356 leader_election.cc:290] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.358618 29357 raft_consensus.cc:515] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.360292 29357 leader_election.cc:290] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.360879 28734 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.361093 28734 ts_tablet_manager.cc:1403] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
W20260320 15:27:29.361583 28239 leader_election.cc:343] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.362183 28736 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.362072 28734 raft_consensus.cc:359] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.362377 28736 ts_tablet_manager.cc:1403] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.003s
I20260320 15:27:29.363106 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa2d48a54dd403c82d7f3a64db5aeab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.363170 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7b53bb7444a39b471218c6039542f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.363482 28583 raft_consensus.cc:3060] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.363921 28736 raft_consensus.cc:359] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.364130 28736 raft_consensus.cc:385] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:29.364260 28239 leader_election.cc:343] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.365479 28736 raft_consensus.cc:740] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.366014 28736 consensus_queue.cc:260] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.366508 28583 raft_consensus.cc:2468] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.362313 28734 raft_consensus.cc:385] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.366721 28734 raft_consensus.cc:740] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.367012 28734 consensus_queue.cc:260] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.368029 28734 ts_tablet_manager.cc:1434] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260320 15:27:29.368140 29290 tablet_loader.cc:96] loaded metadata for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072])
I20260320 15:27:29.368477 28734 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.369041 29290 tablet_loader.cc:96] loaded metadata for tablet 392d04a54aca4d959ef1b94b51382db0 (table default.Table_c424c1cd249645e3bed5b49bdf5f75a1 [id=009b402eeb2f4c2a9bf7d0e94a385ab9])
I20260320 15:27:29.369554 28239 leader_election.cc:304] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:29.369642 29290 tablet_loader.cc:96] loaded metadata for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68 (table default.Table_fda3998f623540e6beca0a6f3455b0b3 [id=9e38178c12314306a0b595904a402f21])
I20260320 15:27:29.370345 29275 raft_consensus.cc:2804] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.370569 29275 raft_consensus.cc:697] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.370988 29275 consensus_queue.cc:237] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.372864 29290 tablet_loader.cc:96] loaded metadata for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6])
I20260320 15:27:29.374188 29290 tablet_loader.cc:96] loaded metadata for tablet 3ed5d220ec4b406aa5bae970f785de63 (table default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c])
I20260320 15:27:29.375613 29290 tablet_loader.cc:96] loaded metadata for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103])
I20260320 15:27:29.376143 29290 tablet_loader.cc:96] loaded metadata for tablet 408f5b4d67514838a17c180ae2cd71df (table default.Table_dfd551c35b134e27a2424da18f1a5bd3 [id=b2921bb259f24e1682ec25e2f083ff5f])
I20260320 15:27:29.376785 29290 tablet_loader.cc:96] loaded metadata for tablet 4150c4c26a26405b9be16f66d15a8948 (table default.Table_12077503f4ba430396cb7392d0e64fa7 [id=18d21a3e06bb4a3f9bb6416cd95047e3])
I20260320 15:27:29.377321 28734 tablet_bootstrap.cc:654] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.383844 28736 ts_tablet_manager.cc:1434] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260320 15:27:29.387161 28736 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.395668 28734 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.395877 28734 ts_tablet_manager.cc:1403] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260320 15:27:29.396965 28734 raft_consensus.cc:359] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.397226 28734 raft_consensus.cc:385] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.397337 28734 raft_consensus.cc:740] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.397591 28734 consensus_queue.cc:260] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.397782 28736 tablet_bootstrap.cc:654] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.398357 28734 ts_tablet_manager.cc:1434] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.398720 28734 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.401142 28734 tablet_bootstrap.cc:654] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.402195 29290 tablet_loader.cc:96] loaded metadata for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_2eff9dcb23c04d50b86d3ad60677435c [id=39d38f795d004300a287cdb264a1f8ca])
I20260320 15:27:29.406008 29241 raft_consensus.cc:515] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.406569 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82c47ae3786401d931bb0b486ca3570" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.406909 28587 raft_consensus.cc:2468] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.408768 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa2d48a54dd403c82d7f3a64db5aeab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.409034 28447 raft_consensus.cc:2468] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.409862 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb7b53bb7444a39b471218c6039542f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.410074 28447 raft_consensus.cc:3060] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.413306 28447 raft_consensus.cc:2468] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.423621 29290 tablet_loader.cc:96] loaded metadata for tablet 44327000792444b89265f4a106eb11d0 (table default.Table_3d0dcd71707248d49f09c2356ee00f6c [id=98dd29a311a548d3bc64a6948ac8743c])
I20260320 15:27:29.424999 29290 tablet_loader.cc:96] loaded metadata for tablet 48c1002cbdd845e2b96dcb57c450cc6d (table default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936])
I20260320 15:27:29.436045 29290 tablet_loader.cc:96] loaded metadata for tablet 48fa326348104d148abbba907f4c547f (table default.Table_28bb9bed388041caa7c69d121e4dbabe [id=f2e791aa9a37498fa24b1904cfc045ea])
I20260320 15:27:29.407778 28240 leader_election.cc:304] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.436746 28240 leader_election.cc:304] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
W20260320 15:27:29.437433 28240 consensus_peers.cc:597] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b03e7555b9a646b599acb0c7dff90c9a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.437628 29356 raft_consensus.cc:2804] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.437721 29356 raft_consensus.cc:493] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.437798 29356 raft_consensus.cc:3060] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.440855 29356 raft_consensus.cc:515] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.441607 29356 leader_election.cc:290] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.441823 29356 raft_consensus.cc:2804] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.442034 29356 raft_consensus.cc:493] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.442152 29356 raft_consensus.cc:3060] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.446460 29356 raft_consensus.cc:515] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.447649 29356 leader_election.cc:290] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.448282 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82c47ae3786401d931bb0b486ca3570" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.448315 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa2d48a54dd403c82d7f3a64db5aeab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.448562 28444 raft_consensus.cc:3060] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.449774 29241 leader_election.cc:290] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.450238 28734 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.450484 28734 ts_tablet_manager.cc:1403] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.052s user 0.028s sys 0.020s
I20260320 15:27:29.450685 29290 tablet_loader.cc:96] loaded metadata for tablet 49654c67435a48448794879464f6cfb2 (table default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a])
I20260320 15:27:29.451097 29290 tablet_loader.cc:96] loaded metadata for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76])
I20260320 15:27:29.451303 29246 raft_consensus.cc:493] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.451402 29290 tablet_loader.cc:96] loaded metadata for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923])
I20260320 15:27:29.451501 29246 raft_consensus.cc:515] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.451730 28297 raft_consensus.cc:2393] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:29.452111 29290 tablet_loader.cc:96] loaded metadata for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69])
I20260320 15:27:29.452400 29246 leader_election.cc:290] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.452497 29290 tablet_loader.cc:96] loaded metadata for tablet 4cfad5cf843a437e8e15c7236d1dc5fe (table default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597])
I20260320 15:27:29.452847 29290 tablet_loader.cc:96] loaded metadata for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f])
I20260320 15:27:29.452956 28444 raft_consensus.cc:2468] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.448559 28447 raft_consensus.cc:3060] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.453825 29290 tablet_loader.cc:96] loaded metadata for tablet 54266dd376164673bffe829dd515b83d (table default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984])
I20260320 15:27:29.454098 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4b53fb135741efad5d3e59a6d4eb3e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.454651 28587 raft_consensus.cc:2468] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.455368 28382 leader_election.cc:304] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.455796 29246 raft_consensus.cc:2804] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.455929 29246 raft_consensus.cc:493] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.456034 29246 raft_consensus.cc:3060] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.456668 28447 raft_consensus.cc:2468] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.457183 28240 leader_election.cc:304] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.457587 29241 raft_consensus.cc:2804] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.457808 28240 leader_election.cc:304] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.458155 29357 raft_consensus.cc:2804] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.458319 29357 raft_consensus.cc:697] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.458379 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.458601 28447 raft_consensus.cc:3060] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.458631 29357 consensus_queue.cc:237] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.459864 29246 raft_consensus.cc:515] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.461930 29246 leader_election.cc:290] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.461863 29241 raft_consensus.cc:697] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.462373 28447 raft_consensus.cc:2468] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.462436 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4b53fb135741efad5d3e59a6d4eb3e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.462769 28587 raft_consensus.cc:3060] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.462991 28240 leader_election.cc:304] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.463303 29356 raft_consensus.cc:2804] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.463644 29356 raft_consensus.cc:697] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.464145 29356 consensus_queue.cc:237] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.464711 28734 raft_consensus.cc:359] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.465014 28734 raft_consensus.cc:385] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.465117 28734 raft_consensus.cc:740] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.465384 28734 consensus_queue.cc:260] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.465497 29241 consensus_queue.cc:237] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.466034 28734 ts_tablet_manager.cc:1434] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260320 15:27:29.466379 28734 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.468281 28734 tablet_bootstrap.cc:654] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.468530 29290 tablet_loader.cc:96] loaded metadata for tablet 547d82cee19b4959b3a779c8b8088e83 (table default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa])
I20260320 15:27:29.468640 28587 raft_consensus.cc:2468] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.468942 29290 tablet_loader.cc:96] loaded metadata for tablet 54a372bb37de43938f0420bdc9c7249c (table default.Table_584780f08e8f4a74b5494b7c0d7ee3d4 [id=6ea6362e6d934bfe8cb54c912b4932dc])
I20260320 15:27:29.469282 29290 tablet_loader.cc:96] loaded metadata for tablet 54c75b9deab44c829f6720480e38c80b (table default.Table_fbb25a9824fc45c7b85abec85ccb75b8 [id=14aef9640e7d427ba93c8b21d7c332c9])
W20260320 15:27:29.469463 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 85392us
I20260320 15:27:29.469739 28382 leader_election.cc:304] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.470376 28736 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.470402 29246 raft_consensus.cc:2804] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.470544 29246 raft_consensus.cc:697] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.470553 28736 ts_tablet_manager.cc:1403] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.084s user 0.009s sys 0.006s
I20260320 15:27:29.470821 29246 consensus_queue.cc:237] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.471350 28736 raft_consensus.cc:359] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.471594 28736 raft_consensus.cc:385] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.471694 28736 raft_consensus.cc:740] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.471724 28383 leader_election.cc:304] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.471794 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4b53fb135741efad5d3e59a6d4eb3e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.471985 28297 raft_consensus.cc:2468] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.471938 28736 consensus_queue.cc:260] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.472337 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.472388 28917 raft_consensus.cc:2749] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.472565 28736 ts_tablet_manager.cc:1434] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:29.472595 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4b53fb135741efad5d3e59a6d4eb3e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.472764 28297 raft_consensus.cc:3060] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.472774 29197 raft_consensus.cc:493] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.472957 28736 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.472949 29197 raft_consensus.cc:515] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.473619 29197 leader_election.cc:290] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:29.474756 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b82c47ae3786401d931bb0b486ca3570" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.474961 28587 raft_consensus.cc:3060] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.475116 28736 tablet_bootstrap.cc:654] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.476089 28297 raft_consensus.cc:2468] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:29.476652 28239 leader_election.cc:343] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.477088 29275 consensus_queue.cc:1048] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:29.477285 28524 leader_election.cc:343] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:29.478757 28587 raft_consensus.cc:2468] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.479936 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9aa2d48a54dd403c82d7f3a64db5aeab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.483345 29290 tablet_loader.cc:96] loaded metadata for tablet 55d089b5a3c74ca4abb53aeb9c487df4 (table default.Table_d4436c4c1a2a4ad1af830b4d62491073 [id=fe749d518f7b425c91fa96254362fb04])
I20260320 15:27:29.485184 29275 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.485435 29275 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.486409 29275 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.487273 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:29.487939 28240 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.498574 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "591ccc1d349e4fd599d5e343f91defe2" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:29.501152 28239 leader_election.cc:343] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.501833 28524 leader_election.cc:304] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.502131 29290 tablet_loader.cc:96] loaded metadata for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4])
I20260320 15:27:29.502812 29197 raft_consensus.cc:2749] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.503077 29299 consensus_queue.cc:1048] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260320 15:27:29.504434 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:29.511601 28239 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.511921 28239 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.513804 29275 raft_consensus.cc:2749] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:29.516709 28240 consensus_peers.cc:597] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1bc86d3d1ff4b3aa8463eb21f619917. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.519024 29246 raft_consensus.cc:493] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.519376 29246 raft_consensus.cc:515] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.520722 29246 leader_election.cc:290] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:29.521637 28382 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:29.526724 29290 tablet_loader.cc:96] loaded metadata for tablet 594357400f8c44138c7c4f11bf6d5afd (table default.Table_b3b4c26f9c844ac9a86b26c019bd713e [id=514f5d793c564cd9b7fd8d1fc84970f8])
I20260320 15:27:29.526739 28733 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.527235 28733 ts_tablet_manager.cc:1403] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.239s user 0.022s sys 0.018s
I20260320 15:27:29.528417 28733 raft_consensus.cc:359] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.528723 28733 raft_consensus.cc:385] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.528849 28733 raft_consensus.cc:740] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.529152 28733 consensus_queue.cc:260] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.536772 28733 ts_tablet_manager.cc:1434] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:29.543148 29290 tablet_loader.cc:96] loaded metadata for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6])
I20260320 15:27:29.543267 28583 raft_consensus.cc:3060] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.547153 29290 tablet_loader.cc:96] loaded metadata for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e])
I20260320 15:27:29.548156 29290 tablet_loader.cc:96] loaded metadata for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4])
W20260320 15:27:29.548465 28383 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:29.548651 28383 leader_election.cc:304] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.548987 29246 raft_consensus.cc:2749] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.554845 28736 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.555357 28736 ts_tablet_manager.cc:1403] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.082s user 0.010s sys 0.069s
I20260320 15:27:29.557590 28734 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.557762 28734 ts_tablet_manager.cc:1403] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.091s user 0.016s sys 0.024s
I20260320 15:27:29.558640 28734 raft_consensus.cc:359] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.558857 28734 raft_consensus.cc:385] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.558969 28734 raft_consensus.cc:740] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.559370 28733 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.559077 28736 raft_consensus.cc:359] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.560227 28736 raft_consensus.cc:385] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.560427 28734 consensus_queue.cc:260] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.560581 28736 raft_consensus.cc:740] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.561002 28734 ts_tablet_manager.cc:1434] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:29.561802 28736 consensus_queue.cc:260] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.563855 28736 ts_tablet_manager.cc:1434] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260320 15:27:29.567175 28734 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.571115 28733 tablet_bootstrap.cc:654] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.574568 28734 tablet_bootstrap.cc:654] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.576102 28736 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.577103 29290 tablet_loader.cc:96] loaded metadata for tablet 60d0b36f462945cbbca7e389edf494d3 (table default.Table_c2a40da69e9d4735b3dd5e71d419b00b [id=cfa06a2552dd47ca947741764a0bf762])
I20260320 15:27:29.577677 29290 tablet_loader.cc:96] loaded metadata for tablet 6197f6a1e1cf4007a481be3d7ccfd425 (table default.Table_19d7fded966f4dfcad98437bdbc0e75c [id=fc0da964b571404bbd36fc2257c95c06])
I20260320 15:27:29.578091 29290 tablet_loader.cc:96] loaded metadata for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c])
I20260320 15:27:29.578166 28736 tablet_bootstrap.cc:654] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.578532 29290 tablet_loader.cc:96] loaded metadata for tablet 6292904782fd4b619483135e58b66684 (table default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4])
I20260320 15:27:29.579471 29241 consensus_queue.cc:1048] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:29.579879 29290 tablet_loader.cc:96] loaded metadata for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (table default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe])
I20260320 15:27:29.580372 29290 tablet_loader.cc:96] loaded metadata for tablet 685e581b3f084d4a9aa97fba0a7767c7 (table default.Table_4068b4f067d74effb3bc0dc5bb1faddc [id=efaa41b4075b4bc98bf0b19fe0c05ba9])
I20260320 15:27:29.581033 29290 tablet_loader.cc:96] loaded metadata for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d])
I20260320 15:27:29.581404 29290 tablet_loader.cc:96] loaded metadata for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d])
I20260320 15:27:29.583591 29245 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.583793 29245 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.585979 29245 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.586323 28736 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.586515 28736 ts_tablet_manager.cc:1403] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260320 15:27:29.586970 28734 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.587152 28734 ts_tablet_manager.cc:1403] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.002s
I20260320 15:27:29.587441 28736 raft_consensus.cc:359] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.587780 28736 raft_consensus.cc:385] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.587884 28736 raft_consensus.cc:740] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.588035 28734 raft_consensus.cc:359] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.588157 28736 consensus_queue.cc:260] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.588330 28734 raft_consensus.cc:385] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.588636 28734 raft_consensus.cc:740] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.588742 28736 ts_tablet_manager.cc:1434] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.588915 28734 consensus_queue.cc:260] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.589059 28736 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.591063 28736 tablet_bootstrap.cc:654] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.594620 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.594985 28734 ts_tablet_manager.cc:1434] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
W20260320 15:27:29.599890 28383 leader_election.cc:343] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.602994 29245 raft_consensus.cc:493] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.603219 29245 raft_consensus.cc:515] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.603994 29245 leader_election.cc:290] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.604525 28734 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.605489 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:29.606086 28383 leader_election.cc:343] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.586483 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.604581 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.606822 28734 tablet_bootstrap.cc:654] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:29.607307 28382 leader_election.cc:343] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.607504 28382 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:29.608045 28382 leader_election.cc:343] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.608227 28382 leader_election.cc:304] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.608515 29156 raft_consensus.cc:2749] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.608645 29290 tablet_loader.cc:96] loaded metadata for tablet 76379bf0fa554820be020f647582a72b (table default.Table_e138abbe7dcc46d1babaa051f60268e7 [id=d84ac38833fe43cfbaefb2f47f7a3af4])
I20260320 15:27:29.608728 29156 raft_consensus.cc:2749] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.609100 29290 tablet_loader.cc:96] loaded metadata for tablet 78288f7aea204870af75060476760ee5 (table default.Table_57819bbc59d24aa1a4d2a28ac5e50728 [id=f74f702b87f24a15bcdca554d3a0bbdd])
I20260320 15:27:29.609269 29156 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.609452 29156 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.610169 29156 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.610451 29290 tablet_loader.cc:96] loaded metadata for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_d021f6d697df4823968c7f3b2d5d779c [id=90afb14544844334aff0e82fa156d6d9])
I20260320 15:27:29.611112 29290 tablet_loader.cc:96] loaded metadata for tablet 7aba58642f7049b1aa0536b6ab6cb271 (table default.Table_d643a0220e4847efb0a5e33127808a91 [id=c759d0cd0eb64c76812102f626211d6d])
I20260320 15:27:29.611071 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.611625 29290 tablet_loader.cc:96] loaded metadata for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_818e1205d4f34148aebe2a4206ca41a5 [id=0998e8ce9720447caf0d5e7fbf145fe7])
W20260320 15:27:29.611804 28382 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.612630 29290 tablet_loader.cc:96] loaded metadata for tablet 7dfbc7338eda489987315e681ee44146 (table default.Table_4a66f1fda9e9438aad057009144db449 [id=20d83b20838641f0ae67bcd3653a0ef4])
I20260320 15:27:29.613214 28736 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.613389 28736 ts_tablet_manager.cc:1403] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.000s
I20260320 15:27:29.613731 28734 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.613917 28734 ts_tablet_manager.cc:1403] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260320 15:27:29.614317 28736 raft_consensus.cc:359] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.614528 28736 raft_consensus.cc:385] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.614634 28736 raft_consensus.cc:740] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.614886 28736 consensus_queue.cc:260] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.615487 28736 ts_tablet_manager.cc:1434] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:29.615849 28736 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.617776 28736 tablet_bootstrap.cc:654] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.619836 28734 raft_consensus.cc:359] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.620153 28734 raft_consensus.cc:385] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.620318 28734 raft_consensus.cc:740] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.620662 28734 consensus_queue.cc:260] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.621618 28734 ts_tablet_manager.cc:1434] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:29.622267 29290 tablet_loader.cc:96] loaded metadata for tablet 7e13c75a376c47c9a0083173865ce4fa (table default.Table_fb580f5b6e1141e1a236805952cf1214 [id=a497706a56ab423cb61065e803c64897])
I20260320 15:27:29.622725 28734 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.625286 28734 tablet_bootstrap.cc:654] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.626588 29290 tablet_loader.cc:96] loaded metadata for tablet 7fb1733e7a0e4774a5be2e06208bb495 (table default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6])
I20260320 15:27:29.627707 29290 tablet_loader.cc:96] loaded metadata for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5 [id=a6da90b5243e4ddc93342b80dac5e784])
I20260320 15:27:29.629185 29290 tablet_loader.cc:96] loaded metadata for tablet 813604f6c6c747eda18f7a01d9d8ec97 (table default.Table_65a8a44c03cd4385a65c2e752446c84c [id=81f681463e614a42b8730cace88a123a])
I20260320 15:27:29.630231 29290 tablet_loader.cc:96] loaded metadata for tablet 825f7e3fdc004fe1a3bcca6deb20cccd (table default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd])
I20260320 15:27:29.631110 29290 tablet_loader.cc:96] loaded metadata for tablet 8428019c3b074b17aec493c14cc2b911 (table default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004])
I20260320 15:27:29.631603 29290 tablet_loader.cc:96] loaded metadata for tablet 85c397eab0ed4df188b0dc2a58077b39 (table default.Table_69342bafde2c42ab8c8eaedb0f28dfb8 [id=667652afa87d46bc878b41081fc53171])
I20260320 15:27:29.632122 29290 tablet_loader.cc:96] loaded metadata for tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e])
I20260320 15:27:29.632601 29290 tablet_loader.cc:96] loaded metadata for tablet 889d620c311a42039c22af53ad0802f2 (table default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306])
I20260320 15:27:29.633960 29290 tablet_loader.cc:96] loaded metadata for tablet 8c4380f86ceb48b8b93eb2a5b9b302c7 (table default.Table_5d4b16da449247bda955fe596e458307 [id=3f666b62287a456ca179d4adffd4dbcb])
I20260320 15:27:29.634122 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.634384 28297 raft_consensus.cc:2468] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.634860 28383 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.635501 29156 raft_consensus.cc:2804] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.635643 29156 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.635713 29290 tablet_loader.cc:96] loaded metadata for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35])
I20260320 15:27:29.635764 29156 raft_consensus.cc:3060] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.636842 29290 tablet_loader.cc:96] loaded metadata for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5])
I20260320 15:27:29.637166 29290 tablet_loader.cc:96] loaded metadata for tablet 8f408aa18d184e5c9a1f900827e70ca8 (table default.Table_ef4a998b01e2437a8edcf95a89e5a314 [id=cf3dff3da9334df6a176702917cfaf18])
I20260320 15:27:29.637553 29290 tablet_loader.cc:96] loaded metadata for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe])
I20260320 15:27:29.638100 29290 tablet_loader.cc:96] loaded metadata for tablet 90cd18e2ccbf48c8b804329bd3abf847 (table default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984])
I20260320 15:27:29.638954 29156 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.641149 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:29.641865 28382 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.642145 29156 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.645771 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.645998 28297 raft_consensus.cc:3060] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.649336 28297 raft_consensus.cc:2468] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.644623 28734 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.651010 28734 ts_tablet_manager.cc:1403] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.006s
I20260320 15:27:29.652141 28734 raft_consensus.cc:359] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.652413 28734 raft_consensus.cc:385] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.652482 28734 raft_consensus.cc:740] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.655928 28734 consensus_queue.cc:260] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.656572 28734 ts_tablet_manager.cc:1434] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:29.663722 28383 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.664273 29156 raft_consensus.cc:2804] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.664440 29156 raft_consensus.cc:697] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.664686 29156 consensus_queue.cc:237] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.666734 28733 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.666942 28733 ts_tablet_manager.cc:1403] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.108s user 0.002s sys 0.001s
I20260320 15:27:29.667882 28733 raft_consensus.cc:359] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.668141 28733 raft_consensus.cc:385] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.668227 28733 raft_consensus.cc:740] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.668486 28733 consensus_queue.cc:260] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.672077 28734 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.674867 28734 tablet_bootstrap.cc:654] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.677124 29290 tablet_loader.cc:96] loaded metadata for tablet 9b838187c21448b6909ad183f9824d2b (table default.Table_602a7434dd7d43a798b06bff668c8d0e [id=cefb97d71bd946b3b75e6d122314bc93])
I20260320 15:27:29.678877 29290 tablet_loader.cc:96] loaded metadata for tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343])
I20260320 15:27:29.679345 29290 tablet_loader.cc:96] loaded metadata for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c])
I20260320 15:27:29.680423 29290 tablet_loader.cc:96] loaded metadata for tablet a17aa9eba95d463b9c0fe5ae7e3cfe13 (table default.Table_a74831451d0047c2a1a67b8b8e2b93d0 [id=ada35f69f6904e7b8d50889e605f0070])
I20260320 15:27:29.681273 29290 tablet_loader.cc:96] loaded metadata for tablet a3bd9c79d2d74e878647f97e66a9bfca (table default.Table_326fcace90ef4e0ba553655d7b3930ae [id=840bfa3d362147309aeded0035f1b3c6])
I20260320 15:27:29.681963 29290 tablet_loader.cc:96] loaded metadata for tablet a76d9688a8ab4b63ae4cf09fd211ebb7 (table default.Table_f051dee79ff04e4faa3f02332b20d6e3 [id=8e3d6498194f4cf08a8d6b1acca86844])
I20260320 15:27:29.682485 29290 tablet_loader.cc:96] loaded metadata for tablet aafd1e123cd043e7828140d2443e047f (table default.Table_134b3b5c79e344f2bf710b12615479be [id=5186e4c77f9f4f59814887534793f0f0])
I20260320 15:27:29.683404 29290 tablet_loader.cc:96] loaded metadata for tablet ad4a30476e7146afae1e77dad078136e (table default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016])
I20260320 15:27:29.685416 29356 raft_consensus.cc:493] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.685612 29356 raft_consensus.cc:515] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.686376 29290 tablet_loader.cc:96] loaded metadata for tablet ae3422cf33324ab89a2b44e27cae0eb2 (table default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b])
I20260320 15:27:29.686393 29356 leader_election.cc:290] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.687132 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c323b7bcb671464ea6ff584a633f10c0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.687438 28447 raft_consensus.cc:2468] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.688011 28240 leader_election.cc:304] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.688374 29356 raft_consensus.cc:2804] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.688426 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c323b7bcb671464ea6ff584a633f10c0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.688524 29356 raft_consensus.cc:493] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.688621 29356 raft_consensus.cc:3060] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.688680 28583 raft_consensus.cc:2468] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.689880 28734 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.690112 28734 ts_tablet_manager.cc:1403] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.003s
I20260320 15:27:29.691191 28734 raft_consensus.cc:359] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.691435 28734 raft_consensus.cc:385] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.691591 28734 raft_consensus.cc:740] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.692104 28733 ts_tablet_manager.cc:1434] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260320 15:27:29.692034 28734 consensus_queue.cc:260] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.696316 29356 raft_consensus.cc:515] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.697584 28734 ts_tablet_manager.cc:1434] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260320 15:27:29.698137 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c323b7bcb671464ea6ff584a633f10c0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.698381 28447 raft_consensus.cc:3060] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.699146 28734 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.701306 28734 tablet_bootstrap.cc:654] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.701529 28447 raft_consensus.cc:2468] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.698573 29290 tablet_loader.cc:96] loaded metadata for tablet b1b6529e062440828fca2e18b129f8e3 (table default.Table_294d5eabb8df40cd888a0769290746bb [id=0350c3a65a5e449dbbdb140b2d82d60a])
I20260320 15:27:29.703346 28240 leader_election.cc:304] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.703743 29241 raft_consensus.cc:2804] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.703990 29241 raft_consensus.cc:697] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.704308 29241 consensus_queue.cc:237] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.704834 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c323b7bcb671464ea6ff584a633f10c0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:29.705272 28583 raft_consensus.cc:3060] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.710467 28733 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.710071 29356 leader_election.cc:290] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.703485 29290 tablet_loader.cc:96] loaded metadata for tablet b20efd94b09746c88bb48ba36af4759e (table default.Table_103c6e9e031144a1bc44f351aa73d587 [id=6a055fea11de4e39addc011ed3a3fe4f])
I20260320 15:27:29.715202 28583 raft_consensus.cc:2468] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.716056 29290 tablet_loader.cc:96] loaded metadata for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd])
I20260320 15:27:29.716951 28733 tablet_bootstrap.cc:654] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.717231 29290 tablet_loader.cc:96] loaded metadata for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0])
I20260320 15:27:29.720379 28736 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.720414 28733 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.720588 28733 ts_tablet_manager.cc:1403] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260320 15:27:29.720608 28736 ts_tablet_manager.cc:1403] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.105s user 0.015s sys 0.016s
I20260320 15:27:29.721493 28733 raft_consensus.cc:359] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.721524 28736 raft_consensus.cc:359] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.721711 28733 raft_consensus.cc:385] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.721741 28736 raft_consensus.cc:385] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.721814 28736 raft_consensus.cc:740] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.721805 28733 raft_consensus.cc:740] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.722084 28736 consensus_queue.cc:260] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.722107 28733 consensus_queue.cc:260] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.722656 28736 ts_tablet_manager.cc:1434] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:29.723052 28736 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.727075 28736 tablet_bootstrap.cc:654] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.731402 28733 ts_tablet_manager.cc:1434] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:29.737010 28734 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.737242 28734 ts_tablet_manager.cc:1403] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.000s
I20260320 15:27:29.737955 29241 raft_consensus.cc:493] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.738198 29241 raft_consensus.cc:515] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.739394 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.739496 29241 leader_election.cc:290] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.739970 28733 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.740546 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.740948 28734 raft_consensus.cc:359] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.741165 28734 raft_consensus.cc:385] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.741237 28734 raft_consensus.cc:740] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:29.741465 28240 leader_election.cc:343] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.741465 28734 consensus_queue.cc:260] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.718040 29290 tablet_loader.cc:96] loaded metadata for tablet bcf4e017e2e743d7b38628d6f1e0d72e (table default.Table_eb8fe7340adc409c98903887ac581604 [id=3b4245a4113946c2838ad27642634e80])
W20260320 15:27:29.742193 28239 leader_election.cc:343] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.742280 28733 tablet_bootstrap.cc:654] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.742363 28239 leader_election.cc:304] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.743016 29241 raft_consensus.cc:2749] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.744678 28736 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.744845 28736 ts_tablet_manager.cc:1403] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260320 15:27:29.745749 28736 raft_consensus.cc:359] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.745961 28736 raft_consensus.cc:385] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.746050 28736 raft_consensus.cc:740] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.746364 28736 consensus_queue.cc:260] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.746690 28734 ts_tablet_manager.cc:1434] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260320 15:27:29.746855 29290 tablet_loader.cc:96] loaded metadata for tablet c3a49e310b524f3b87eb89f3e84b7800 (table default.Table_536e57665fa14e0e915512c1a27ab8fe [id=f2497b3b3f73471e872d6cba3e653118])
I20260320 15:27:29.747057 28736 ts_tablet_manager.cc:1434] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:29.749217 29290 tablet_loader.cc:96] loaded metadata for tablet c7e3a6fa7dc74de286177a285dd3d877 (table default.Table_094bdcada4784500a661a56536bb221b [id=5bed4e07a5c04c2f8950f42423d35bbe])
I20260320 15:27:29.749866 28734 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.751873 28736 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.753973 28736 tablet_bootstrap.cc:654] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.761955 29290 tablet_loader.cc:96] loaded metadata for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2])
I20260320 15:27:29.762645 29290 tablet_loader.cc:96] loaded metadata for tablet cc97a5f2bd964a05bb7b2427226debdd (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06])
I20260320 15:27:29.763310 28734 tablet_bootstrap.cc:654] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.764196 29290 tablet_loader.cc:96] loaded metadata for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191])
I20260320 15:27:29.765784 29290 tablet_loader.cc:96] loaded metadata for tablet cea3bf14653f4555acfed1577a79ee4f (table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568])
I20260320 15:27:29.766705 29290 tablet_loader.cc:96] loaded metadata for tablet d5095b42b69d40539118c1f63a95f795 (table default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96])
I20260320 15:27:29.767673 29290 tablet_loader.cc:96] loaded metadata for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c])
I20260320 15:27:29.768733 28733 tablet_bootstrap.cc:492] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.768921 28733 ts_tablet_manager.cc:1403] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.003s
I20260320 15:27:29.768775 29290 tablet_loader.cc:96] loaded metadata for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617])
I20260320 15:27:29.769526 29290 tablet_loader.cc:96] loaded metadata for tablet d87a2f5513684da79f6eebbed89d0392 (table default.Table_a2607c8356f048f1bb3fbe1f240848c2 [id=713c4ac167384536b97796e4ba84e04e])
I20260320 15:27:29.769790 28733 raft_consensus.cc:359] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.770000 28733 raft_consensus.cc:385] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.770080 28733 raft_consensus.cc:740] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.770373 29290 tablet_loader.cc:96] loaded metadata for tablet da29d2ffcb07428789c22a573d680289 (table default.Table_0f2d272a74ac4a8ab851d662826ba9e6 [id=611935c42b0e42c7b8b6fa719c526910])
I20260320 15:27:29.770391 28733 consensus_queue.cc:260] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.771394 29290 tablet_loader.cc:96] loaded metadata for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e])
I20260320 15:27:29.772688 28736 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.772945 28736 ts_tablet_manager.cc:1403] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.005s
I20260320 15:27:29.773514 29290 tablet_loader.cc:96] loaded metadata for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677])
I20260320 15:27:29.773856 28736 raft_consensus.cc:359] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.774058 28736 raft_consensus.cc:385] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.774129 28736 raft_consensus.cc:740] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.774377 28736 consensus_queue.cc:260] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.774931 28736 ts_tablet_manager.cc:1434] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:29.775300 28736 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.777330 28736 tablet_bootstrap.cc:654] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.778383 29241 raft_consensus.cc:493] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.778519 29241 raft_consensus.cc:515] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.779183 29241 leader_election.cc:290] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.779395 28733 ts_tablet_manager.cc:1434] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:29.781045 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:29.781687 28240 leader_election.cc:343] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.783159 28733 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.784636 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.785938 28733 tablet_bootstrap.cc:654] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:29.788410 28239 leader_election.cc:343] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.788606 28239 leader_election.cc:304] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.789043 29241 raft_consensus.cc:2749] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.789829 28734 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.790006 28734 ts_tablet_manager.cc:1403] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.004s
I20260320 15:27:29.790915 28734 raft_consensus.cc:359] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.791121 28734 raft_consensus.cc:385] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.791213 28734 raft_consensus.cc:740] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.791450 28734 consensus_queue.cc:260] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.792801 28736 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.792955 28736 ts_tablet_manager.cc:1403] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260320 15:27:29.793375 28734 ts_tablet_manager.cc:1434] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:29.793993 29290 tablet_loader.cc:96] loaded metadata for tablet dd315b5d49104d679b03515b65ee30ac (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2])
I20260320 15:27:29.793905 28736 raft_consensus.cc:359] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.794098 28736 raft_consensus.cc:385] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.794291 28736 raft_consensus.cc:740] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.794713 28736 consensus_queue.cc:260] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.795110 29290 tablet_loader.cc:96] loaded metadata for tablet e0e7ae1ff302428abf6cc592bbca0aa9 (table default.Table_2827438ad3b74f42968ceb52cf633bff [id=f1748199619b4638917812d77feed8f3])
I20260320 15:27:29.795747 28736 ts_tablet_manager.cc:1434] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260320 15:27:29.796757 28734 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.797175 28736 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.799815 28736 tablet_bootstrap.cc:654] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.802640 28734 tablet_bootstrap.cc:654] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.807425 29290 tablet_loader.cc:96] loaded metadata for tablet e13f1a65661949258f521dc847e674cd (table default.Table_0aeb446eb4d34a5ca84cd76d2b01b76c [id=7dc7c07f0ba646fbb6e8ac8b228e6e5d])
I20260320 15:27:29.808005 29290 tablet_loader.cc:96] loaded metadata for tablet e2243e6811744b7cba99a487f8e8bb83 (table default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356])
I20260320 15:27:29.812458 29290 tablet_loader.cc:96] loaded metadata for tablet e5479578a7cd470a9a4851d252a9e93e (table default.Table_ab8ef4acb1c64a06adb7b895d3e5cf47 [id=0a1df577ad734c34b6cfe3cba679ed7f])
I20260320 15:27:29.814086 29290 tablet_loader.cc:96] loaded metadata for tablet ea7f260f4edd4fc082ab82880143fa31 (table default.Table_1642210236684e85bbcfdf569f9ae5d6 [id=4a69c667d48e4ae0b73e59fd343e4ff4])
I20260320 15:27:29.814656 29241 raft_consensus.cc:493] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.814854 29241 raft_consensus.cc:515] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.815034 29290 tablet_loader.cc:96] loaded metadata for tablet eb011d672f244585b90272443897ed5a (table default.Table_186af3c003da48d78f6c983d6f7b63d8 [id=8f5be23c174e487dacf40ad504ead589])
I20260320 15:27:29.815827 29241 leader_election.cc:290] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.815500 29290 tablet_loader.cc:96] loaded metadata for tablet ec6961acec774d2f9161f057f64b3fdf (table default.Table_089d0cb58bd1495cbe761dae3b90b562 [id=8fe18e7035b544b3bd5b5ef3da3bd504])
I20260320 15:27:29.816661 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.817045 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.817413 29290 tablet_loader.cc:96] loaded metadata for tablet f05205970cd249b68d138b904ba61c9e (table default.Table_a0fc0f5fd6874fa4b8e5796c82a549a8 [id=2d0fe40191c9403ab71159ef1377e21c])
W20260320 15:27:29.817694 28240 leader_election.cc:343] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.817848 29290 tablet_loader.cc:96] loaded metadata for tablet f0a31d385b81456fb46ea52083874fcf (table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b])
W20260320 15:27:29.817878 28239 leader_election.cc:343] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.818007 28240 leader_election.cc:304] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.818498 29290 tablet_loader.cc:96] loaded metadata for tablet f3e4a749f0744e68b502af97edeeb584 (table default.Table_c19d1d44b172435dbb61e970e1b3ced2 [id=c825e1c06ebc423c98aadd08b90bfdc0])
I20260320 15:27:29.819003 29241 raft_consensus.cc:2749] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.820415 28733 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.820598 28736 tablet_bootstrap.cc:492] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.820606 28733 ts_tablet_manager.cc:1403] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.003s
I20260320 15:27:29.820854 28736 ts_tablet_manager.cc:1403] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.002s
I20260320 15:27:29.820891 28734 tablet_bootstrap.cc:492] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.821067 28734 ts_tablet_manager.cc:1403] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.002s
I20260320 15:27:29.821556 28733 raft_consensus.cc:359] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.821782 28733 raft_consensus.cc:385] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.821904 28733 raft_consensus.cc:740] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.821959 28734 raft_consensus.cc:359] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.822116 28736 raft_consensus.cc:359] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.822188 28733 consensus_queue.cc:260] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.822288 28734 raft_consensus.cc:385] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.822373 28734 raft_consensus.cc:740] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.822388 28736 raft_consensus.cc:385] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.822489 28736 raft_consensus.cc:740] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.822701 28734 consensus_queue.cc:260] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.822839 28736 consensus_queue.cc:260] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.827566 28736 ts_tablet_manager.cc:1434] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:29.827970 28736 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:29.829766 28524 consensus_peers.cc:597] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.830093 28736 tablet_bootstrap.cc:654] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.831009 28734 ts_tablet_manager.cc:1434] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:29.831671 28734 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.835386 28734 tablet_bootstrap.cc:654] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.836987 29290 tablet_loader.cc:96] loaded metadata for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_3979e77f6aba4d9f80ba813c92287b84 [id=4737d6ecec7041afb0192ab248a093ca])
I20260320 15:27:29.840068 29290 tablet_loader.cc:96] loaded metadata for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e])
I20260320 15:27:29.841142 28733 ts_tablet_manager.cc:1434] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.020s user 0.002s sys 0.001s
I20260320 15:27:29.841205 29290 tablet_loader.cc:96] loaded metadata for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4])
I20260320 15:27:29.841878 29290 tablet_loader.cc:96] loaded metadata for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc])
I20260320 15:27:29.842969 29290 tablet_loader.cc:96] loaded metadata for tablet fcff253b9cce45b4ad11dc38397afee2 (table default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18])
W20260320 15:27:29.843901 29290 catalog_manager.cc:1486] Time spent T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Loading table and tablet metadata into memory: real 1.144s user 0.191s sys 0.038s
I20260320 15:27:29.844223 29290 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260320 15:27:29.845983 29156 raft_consensus.cc:493] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.846180 29156 raft_consensus.cc:515] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.846472 28733 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.846894 29156 leader_election.cc:290] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.848284 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:29.848740 28733 tablet_bootstrap.cc:654] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.849375 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:29.850062 28382 leader_election.cc:343] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:29.850581 28383 leader_election.cc:343] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.850716 28383 leader_election.cc:304] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.851236 29156 raft_consensus.cc:2749] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.852317 28734 tablet_bootstrap.cc:492] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.852481 28734 ts_tablet_manager.cc:1403] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260320 15:27:29.853344 28734 raft_consensus.cc:359] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.853523 28734 raft_consensus.cc:385] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.853607 28734 raft_consensus.cc:740] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.853816 28734 consensus_queue.cc:260] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.854354 28734 ts_tablet_manager.cc:1434] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.854688 28734 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.858821 29290 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:29.861065 28736 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.861478 28736 ts_tablet_manager.cc:1403] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.004s
I20260320 15:27:29.862496 28736 raft_consensus.cc:359] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.862773 28736 raft_consensus.cc:385] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.862915 28736 raft_consensus.cc:740] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.864192 28736 consensus_queue.cc:260] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.868108 29290 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260320 15:27:29.871526 28734 tablet_bootstrap.cc:654] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.872640 28736 ts_tablet_manager.cc:1434] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:29.875597 29156 raft_consensus.cc:493] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.875804 29156 raft_consensus.cc:515] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.876696 29156 leader_election.cc:290] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.877818 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc97a5f2bd964a05bb7b2427226debdd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:29.878573 28382 leader_election.cc:343] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.879405 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc97a5f2bd964a05bb7b2427226debdd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:29.879668 28297 raft_consensus.cc:2468] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:29.880726 28383 leader_election.cc:304] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.881506 29156 raft_consensus.cc:2804] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.881675 29156 raft_consensus.cc:493] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.881804 29156 raft_consensus.cc:3060] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.887404 29290 catalog_manager.cc:1514] Loading token signing keys...
I20260320 15:27:29.888586 28736 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.891119 28736 tablet_bootstrap.cc:654] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.897451 29245 consensus_queue.cc:1048] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260320 15:27:29.897739 28733 tablet_bootstrap.cc:492] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.897989 28733 ts_tablet_manager.cc:1403] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.052s user 0.017s sys 0.001s
I20260320 15:27:29.899125 28733 raft_consensus.cc:359] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.899434 28733 raft_consensus.cc:385] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.899524 28733 raft_consensus.cc:740] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.899874 28733 consensus_queue.cc:260] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.900569 28733 ts_tablet_manager.cc:1434] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:29.901036 28733 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.903506 28733 tablet_bootstrap.cc:654] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.905592 29156 raft_consensus.cc:515] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.905702 29245 consensus_queue.cc:1048] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:29.916966 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc97a5f2bd964a05bb7b2427226debdd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:29.917291 28298 raft_consensus.cc:3060] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.918284 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc97a5f2bd964a05bb7b2427226debdd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:29.918933 28382 leader_election.cc:343] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.919621 29156 leader_election.cc:290] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.922492 28298 raft_consensus.cc:2468] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:29.923249 28383 leader_election.cc:304] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.923606 28733 tablet_bootstrap.cc:492] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:29.923780 28733 ts_tablet_manager.cc:1403] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.004s
I20260320 15:27:29.923965 29156 raft_consensus.cc:2804] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.924801 29156 raft_consensus.cc:697] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:29.925439 29156 consensus_queue.cc:237] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.925678 28733 raft_consensus.cc:359] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.925892 28733 raft_consensus.cc:385] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.926023 28733 raft_consensus.cc:740] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.926330 28733 consensus_queue.cc:260] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.926964 28733 ts_tablet_manager.cc:1434] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:29.927850 28733 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:29.930048 28733 tablet_bootstrap.cc:654] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.932233 29290 catalog_manager.cc:6055] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Loaded TSK: 0
I20260320 15:27:29.933125 28736 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:29.933341 28736 ts_tablet_manager.cc:1403] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.001s
I20260320 15:27:29.933791 28734 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:29.933984 28734 ts_tablet_manager.cc:1403] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.079s user 0.013s sys 0.016s
I20260320 15:27:29.934340 28736 raft_consensus.cc:359] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.934554 28736 raft_consensus.cc:385] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.934671 29290 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260320 15:27:29.934774 28734 raft_consensus.cc:359] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.934984 28734 raft_consensus.cc:385] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:29.935081 28734 raft_consensus.cc:740] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.935127 29241 raft_consensus.cc:493] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.937366 29241 raft_consensus.cc:515] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.937994 29241 leader_election.cc:290] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:29.939319 28240 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:29.947486 28917 raft_consensus.cc:493] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.947808 28917 raft_consensus.cc:515] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.948552 28917 leader_election.cc:290] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.949951 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:29.950758 28382 leader_election.cc:343] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:29.951923 28239 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:29.952095 28239 leader_election.cc:304] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:29.952879 29241 raft_consensus.cc:2749] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.959664 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:29.960415 28383 leader_election.cc:343] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.960842 28383 leader_election.cc:304] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:29.963937 28917 raft_consensus.cc:2749] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:29.936354 28734 consensus_queue.cc:260] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.968825 28734 ts_tablet_manager.cc:1434] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.035s user 0.001s sys 0.001s
I20260320 15:27:29.969462 28734 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:29.969816 28583 raft_consensus.cc:3060] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.972306 28734 tablet_bootstrap.cc:654] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:29.973261 28901 master_service.cc:954] ReplaceTablet: received request to replace tablet 889d620c311a42039c22af53ad0802f2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:29.974402 29241 raft_consensus.cc:493] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:29.974583 29241 raft_consensus.cc:515] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.975517 29241 leader_election.cc:290] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:29.976001 28917 consensus_queue.cc:1048] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:29.976995 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:29.977310 28447 raft_consensus.cc:2468] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:29.978013 28240 leader_election.cc:304] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.978542 29241 raft_consensus.cc:2804] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:29.978729 29241 raft_consensus.cc:493] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:29.978876 29241 raft_consensus.cc:3060] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.983507 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:29.984354 28239 leader_election.cc:343] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:29.985265 28890 master_service.cc:954] ReplaceTablet: received request to replace tablet 26818e620ff649d18b671e4f4c7c7e74 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:29.985714 29241 raft_consensus.cc:515] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.986644 28890 master_service.cc:954] ReplaceTablet: received request to replace tablet 583ed053b2a742c7b029f348f4495101 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:29.988359 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:29.988693 28447 raft_consensus.cc:3060] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:29.990000 28892 master_service.cc:954] ReplaceTablet: received request to replace tablet 6231fe30c5b34d0bb7a233abbffeb1e8 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:29.990860 28889 master_service.cc:954] ReplaceTablet: received request to replace tablet 2ee4a0d559ce4dfea878bd3f1c03c09a from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:29.991475 28383 consensus_peers.cc:597] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2355eadcae6483bae7ae475d74dd131. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:29.934638 28736 raft_consensus.cc:740] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:29.991950 28736 consensus_queue.cc:260] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.993727 28447 raft_consensus.cc:2468] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:29.994491 28240 leader_election.cc:304] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:29.994943 29356 raft_consensus.cc:2804] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:29.995155 29356 raft_consensus.cc:697] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:29.995611 29356 consensus_queue.cc:237] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:29.983753 28903 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_c424c1cd249645e3bed5b49bdf5f75a1" } modify_external_catalogs: true
I20260320 15:27:29.996351 28903 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_c424c1cd249645e3bed5b49bdf5f75a1" } modify_external_catalogs: true
I20260320 15:27:29.995167 29275 consensus_queue.cc:1048] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260320 15:27:29.984956 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2efdb3d4fafe437d8d4e4bedb4f791a5" } new_table_name: "default.Table_29cefe9b80e84240960c1e41f7401aaa" modify_external_catalogs: true
I20260320 15:27:29.987577 28900 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f051dee79ff04e4faa3f02332b20d6e3" } modify_external_catalogs: true
I20260320 15:27:30.002151 28900 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f051dee79ff04e4faa3f02332b20d6e3" } modify_external_catalogs: true
I20260320 15:27:30.004395 29275 consensus_queue.cc:1048] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.004681 28736 ts_tablet_manager.cc:1434] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.071s user 0.002s sys 0.000s
I20260320 15:27:30.005115 28736 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:29.986272 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2eff9dcb23c04d50b86d3ad60677435c" } new_table_name: "default.Table_923e214b36464c938d8b5be225e0a94b" modify_external_catalogs: true
I20260320 15:27:29.984231 28905 catalog_manager.cc:5671] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } }
I20260320 15:27:30.006471 28890 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d4436c4c1a2a4ad1af830b4d62491073" } modify_external_catalogs: true
I20260320 15:27:30.008118 28736 tablet_bootstrap.cc:654] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.008985 28890 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d4436c4c1a2a4ad1af830b4d62491073" } modify_external_catalogs: true
I20260320 15:27:30.013424 28917 consensus_queue.cc:1048] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:29.973372 28902 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_7879fae3c7d843cbaebc6ccd995af608" } modify_external_catalogs: true
I20260320 15:27:30.015373 28902 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_7879fae3c7d843cbaebc6ccd995af608" } modify_external_catalogs: true
I20260320 15:27:30.023624 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.61:44739
--webserver_interface=127.27.74.61
--webserver_port=39103
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260320 15:27:30.035671 28917 consensus_queue.cc:1048] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.038266 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.060379 28734 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:30.060563 28734 ts_tablet_manager.cc:1403] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.091s user 0.003s sys 0.003s
I20260320 15:27:30.062597 28733 tablet_bootstrap.cc:492] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.062801 28733 ts_tablet_manager.cc:1403] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.135s user 0.026s sys 0.061s
I20260320 15:27:30.063889 28733 raft_consensus.cc:359] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.064304 28733 raft_consensus.cc:385] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.064401 28733 raft_consensus.cc:740] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.064664 28733 consensus_queue.cc:260] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.064957 28734 raft_consensus.cc:359] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.065143 28734 raft_consensus.cc:385] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.065274 28734 raft_consensus.cc:740] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.065513 28734 consensus_queue.cc:260] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.066102 28734 ts_tablet_manager.cc:1434] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:30.020740 29275 raft_consensus.cc:493] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.067553 29275 raft_consensus.cc:515] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.068280 29275 leader_election.cc:290] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.068426 29275 raft_consensus.cc:493] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.068581 29275 raft_consensus.cc:515] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.069082 28733 ts_tablet_manager.cc:1434] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:30.069180 29275 leader_election.cc:290] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.069453 28733 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.031319 29357 raft_consensus.cc:493] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.069641 29357 raft_consensus.cc:515] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.070155 29357 leader_election.cc:290] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:30.039129 28239 leader_election.cc:343] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.071403 28733 tablet_bootstrap.cc:654] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.066444 28734 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:30.075106 28917 consensus_queue.cc:1048] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.077679 28736 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:30.077921 28736 ts_tablet_manager.cc:1403] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.073s user 0.004s sys 0.000s
I20260320 15:27:30.078788 28736 raft_consensus.cc:359] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.079058 28736 raft_consensus.cc:385] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.079203 28736 raft_consensus.cc:740] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.079560 28736 consensus_queue.cc:260] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.080349 28736 ts_tablet_manager.cc:1434] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:30.081233 28736 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:30.084918 28736 tablet_bootstrap.cc:654] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.075382 28734 tablet_bootstrap.cc:654] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.091629 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.091781 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7b964395f24e45bc2715be3fac6dbe" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.091924 28444 raft_consensus.cc:2468] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.091941 28447 raft_consensus.cc:2468] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.095300 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.095461 28240 leader_election.cc:304] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.095568 28444 raft_consensus.cc:2468] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.095999 28240 leader_election.cc:304] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.096272 29429 raft_consensus.cc:2804] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.096372 29429 raft_consensus.cc:493] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.096480 29429 raft_consensus.cc:3060] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.096570 29299 consensus_queue.cc:1048] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.096769 29246 raft_consensus.cc:493] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.096932 29246 raft_consensus.cc:515] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.097609 29430 raft_consensus.cc:2804] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.097719 29430 raft_consensus.cc:493] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.097862 29430 raft_consensus.cc:3060] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.098198 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6201b74fdb0428e98f2eebb0b24d01a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.098441 28300 raft_consensus.cc:2468] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.099043 28383 leader_election.cc:304] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.099341 28917 raft_consensus.cc:2804] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.099452 28917 raft_consensus.cc:493] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.099602 28917 raft_consensus.cc:3060] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.099861 29429 raft_consensus.cc:515] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.100610 29429 leader_election.cc:290] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.101986 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.102134 28736 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:30.102382 28736 ts_tablet_manager.cc:1403] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.000s
I20260320 15:27:30.102494 28586 raft_consensus.cc:3060] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.102823 28917 raft_consensus.cc:515] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.103334 28736 raft_consensus.cc:359] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.103570 28736 raft_consensus.cc:385] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.103695 28736 raft_consensus.cc:740] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.104029 28736 consensus_queue.cc:260] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.104223 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6201b74fdb0428e98f2eebb0b24d01a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.104393 28300 raft_consensus.cc:3060] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.104951 28736 ts_tablet_manager.cc:1434] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:30.105382 28736 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:30.107005 28586 raft_consensus.cc:2468] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.107371 28300 raft_consensus.cc:2468] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.107438 28736 tablet_bootstrap.cc:654] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.109185 28239 leader_election.cc:304] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.109200 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.109511 29429 raft_consensus.cc:2804] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.109676 29429 raft_consensus.cc:697] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
W20260320 15:27:30.109743 28239 leader_election.cc:343] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.109977 29429 consensus_queue.cc:237] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.111620 28240 leader_election.cc:304] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.111902 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.112121 28446 raft_consensus.cc:3060] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.112340 29357 raft_consensus.cc:2804] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.112432 29357 raft_consensus.cc:493] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.112531 29357 raft_consensus.cc:3060] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.115242 28446 raft_consensus.cc:2468] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.115310 29357 raft_consensus.cc:515] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.116159 28383 leader_election.cc:304] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.117002 29357 leader_election.cc:290] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.118258 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:30.119879 28239 leader_election.cc:343] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.121096 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.121804 28446 raft_consensus.cc:3060] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.121933 29430 raft_consensus.cc:515] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.123239 29430 leader_election.cc:290] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.123693 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7b964395f24e45bc2715be3fac6dbe" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.040524 29241 leader_election.cc:290] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.123970 28584 raft_consensus.cc:3060] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.126526 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7b964395f24e45bc2715be3fac6dbe" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.126770 28447 raft_consensus.cc:3060] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.128264 28584 raft_consensus.cc:2468] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.129040 29156 raft_consensus.cc:2804] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.129031 28239 leader_election.cc:304] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.130290 28447 raft_consensus.cc:2468] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.131853 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6201b74fdb0428e98f2eebb0b24d01a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.132345 29245 consensus_queue.cc:1048] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.132870 28917 leader_election.cc:290] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:30.134357 28382 leader_election.cc:343] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.137337 29246 leader_election.cc:290] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.137909 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6201b74fdb0428e98f2eebb0b24d01a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.138173 29241 raft_consensus.cc:2804] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.138384 29241 raft_consensus.cc:697] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.138427 29357 raft_consensus.cc:493] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.138576 29357 raft_consensus.cc:515] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.138723 29241 consensus_queue.cc:237] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.139545 29357 leader_election.cc:290] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:30.138813 28382 leader_election.cc:343] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.142735 28446 raft_consensus.cc:2468] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.143472 28240 leader_election.cc:304] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.144238 29429 raft_consensus.cc:2804] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.144455 29429 raft_consensus.cc:697] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.144990 29429 consensus_queue.cc:237] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.147238 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fa326348104d148abbba907f4c547f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.147636 28586 raft_consensus.cc:2468] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.148257 28239 leader_election.cc:304] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.148696 29356 raft_consensus.cc:2804] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.148949 29356 raft_consensus.cc:493] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.149137 28734 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:30.149643 29356 raft_consensus.cc:3060] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.150585 28734 ts_tablet_manager.cc:1403] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.002s
I20260320 15:27:30.151932 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e7b964395f24e45bc2715be3fac6dbe" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.152232 28587 raft_consensus.cc:2376] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1. Re-sending same reply.
I20260320 15:27:30.151002 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fa326348104d148abbba907f4c547f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.154587 28446 raft_consensus.cc:2468] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.156000 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27ccc693e7704650bd77e0c4ca11862e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.156293 28585 raft_consensus.cc:2376] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1. Re-sending same reply.
I20260320 15:27:30.157610 29430 raft_consensus.cc:493] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.157953 29430 raft_consensus.cc:515] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.159060 29430 leader_election.cc:290] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.160288 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.161119 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.161216 28446 raft_consensus.cc:2468] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.161664 28240 leader_election.cc:304] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:30.161886 28239 leader_election.cc:343] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.162094 29430 raft_consensus.cc:2804] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.162096 29156 raft_consensus.cc:697] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:30.162186 29430 raft_consensus.cc:493] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.162308 29430 raft_consensus.cc:3060] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.160601 28734 raft_consensus.cc:359] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.162659 28734 raft_consensus.cc:385] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.162782 28734 raft_consensus.cc:740] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.162534 29156 consensus_queue.cc:237] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.166882 29356 raft_consensus.cc:515] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.169155 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fa326348104d148abbba907f4c547f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.169533 28585 raft_consensus.cc:3060] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.172645 29356 leader_election.cc:290] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.173606 29446 consensus_queue.cc:1048] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.173990 28736 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:30.174141 28736 ts_tablet_manager.cc:1403] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.010s
I20260320 15:27:30.176622 28585 raft_consensus.cc:2468] T 48fa326348104d148abbba907f4c547f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.178225 28736 raft_consensus.cc:359] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.178588 28736 raft_consensus.cc:385] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.178776 28736 raft_consensus.cc:740] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.179087 28736 consensus_queue.cc:260] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.179776 28736 ts_tablet_manager.cc:1434] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:30.182526 28239 leader_election.cc:304] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.182855 28736 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:30.189101 29430 raft_consensus.cc:515] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:30.191654 28906 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
I20260320 15:27:30.189781 28736 tablet_bootstrap.cc:654] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.197240 29430 leader_election.cc:290] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.197477 29430 raft_consensus.cc:2804] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.197652 29430 raft_consensus.cc:697] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.197472 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:30.198333 28239 leader_election.cc:343] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.199425 29430 consensus_queue.cc:237] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.211764 28733 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.211992 28733 ts_tablet_manager.cc:1403] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.143s user 0.026s sys 0.037s
I20260320 15:27:30.212975 28733 raft_consensus.cc:359] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.213207 28733 raft_consensus.cc:385] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.213310 28733 raft_consensus.cc:740] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.213663 28733 consensus_queue.cc:260] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.221278 28733 ts_tablet_manager.cc:1434] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:30.221714 28733 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.230501 28733 tablet_bootstrap.cc:654] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.236840 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 889d620c311a42039c22af53ad0802f2
I20260320 15:27:30.238725 28901 catalog_manager.cc:6741] ReplaceTablet: tablet 889d620c311a42039c22af53ad0802f2 deleted and replaced by tablet 7f94140a5e8b4a06890bca86456d64a6
I20260320 15:27:30.243685 29446 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.243901 29446 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.244673 29446 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.245668 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.245925 28297 raft_consensus.cc:2468] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:30.246766 28524 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.247371 29449 raft_consensus.cc:2804] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.247481 29449 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.247607 29449 raft_consensus.cc:3060] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.250931 29449 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.253963 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.254231 28297 raft_consensus.cc:3060] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.257956 28297 raft_consensus.cc:2468] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:30.258846 28524 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:30.259320 29446 raft_consensus.cc:2804] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.259512 29446 raft_consensus.cc:697] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:30.259883 29446 consensus_queue.cc:237] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.261574 29449 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.264909 28733 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.265443 28733 ts_tablet_manager.cc:1403] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.044s user 0.013s sys 0.001s
I20260320 15:27:30.266793 28733 raft_consensus.cc:359] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.267366 28733 raft_consensus.cc:385] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.267604 28733 raft_consensus.cc:740] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.268051 28733 consensus_queue.cc:260] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.269033 28733 ts_tablet_manager.cc:1434] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:30.269455 28733 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.271097 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48fa326348104d148abbba907f4c547f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.271335 28444 raft_consensus.cc:3060] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.274345 28734 consensus_queue.cc:260] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.277931 28736 tablet_bootstrap.cc:492] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:30.278160 28736 ts_tablet_manager.cc:1403] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.095s user 0.009s sys 0.001s
I20260320 15:27:30.279068 28736 raft_consensus.cc:359] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.279301 28736 raft_consensus.cc:385] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.279368 28736 raft_consensus.cc:740] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.280862 28736 consensus_queue.cc:260] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.283711 29356 raft_consensus.cc:493] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.283910 29356 raft_consensus.cc:515] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.284401 28733 tablet_bootstrap.cc:654] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.284727 29356 leader_election.cc:290] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.284916 29356 consensus_queue.cc:1048] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.289012 29241 raft_consensus.cc:493] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.289252 29241 raft_consensus.cc:515] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.290066 29241 leader_election.cc:290] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.291450 28736 ts_tablet_manager.cc:1434] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260320 15:27:30.291438 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_8b5b70bee34f4caabd4a276b20f57793"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.293007 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b5b70bee34f4caabd4a276b20f57793 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.293144 29246 raft_consensus.cc:493] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.293906 28736 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:30.296608 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:30.296649 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 176026us
I20260320 15:27:30.297458 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.298835 28901 catalog_manager.cc:5671] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
W20260320 15:27:30.299077 28239 leader_election.cc:343] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:30.299650 28239 leader_election.cc:343] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.300221 29241 consensus_queue.cc:1048] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.297133 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e2c5841a2d914105ac376afba3c85cb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:30.300444 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:30.300945 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2c5841a2d914105ac376afba3c85cb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.302007 28733 tablet_bootstrap.cc:492] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.302204 28733 ts_tablet_manager.cc:1403] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.009s
I20260320 15:27:30.303553 28733 raft_consensus.cc:359] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.303849 28733 raft_consensus.cc:385] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.303987 28733 raft_consensus.cc:740] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.304315 28733 consensus_queue.cc:260] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.304894 28733 ts_tablet_manager.cc:1434] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:30.305266 28733 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.307436 28733 tablet_bootstrap.cc:654] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.308811 29444 raft_consensus.cc:493] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.309075 29444 raft_consensus.cc:515] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.309571 28903 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 392d04a54aca4d959ef1b94b51382db0
I20260320 15:27:30.310256 29444 leader_election.cc:290] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.310205 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_38d617af0e3e4149b252aac03d1863ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.310861 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38d617af0e3e4149b252aac03d1863ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.311208 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:30.311825 28382 leader_election.cc:343] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:30.316525 28904 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4190329df1b644c392ff8cbace9aae7a has no consensus state
W20260320 15:27:30.317391 28873 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
I20260320 15:27:30.317698 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:30.319123 28446 raft_consensus.cc:3060] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.321815 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 6231fe30c5b34d0bb7a233abbffeb1e8
I20260320 15:27:30.322376 28892 catalog_manager.cc:6741] ReplaceTablet: tablet 6231fe30c5b34d0bb7a233abbffeb1e8 deleted and replaced by tablet e779a274913f4caeaf82e3f180457830
I20260320 15:27:30.322678 28446 raft_consensus.cc:2468] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.323155 28240 leader_election.cc:304] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.323365 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:30.323833 28889 catalog_manager.cc:6741] ReplaceTablet: tablet 2ee4a0d559ce4dfea878bd3f1c03c09a deleted and replaced by tablet fca60a0f372f4d1194d754d124b78ff8
I20260320 15:27:30.324163 29241 raft_consensus.cc:2804] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.324404 29241 raft_consensus.cc:697] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.324765 29241 consensus_queue.cc:237] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.327016 29246 raft_consensus.cc:515] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.329468 28733 tablet_bootstrap.cc:492] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.329732 28733 ts_tablet_manager.cc:1403] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20260320 15:27:30.329120 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.330919 28733 raft_consensus.cc:359] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.331369 28733 raft_consensus.cc:385] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.331580 28733 raft_consensus.cc:740] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.332091 28733 consensus_queue.cc:260] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.332721 28900 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet a76d9688a8ab4b63ae4cf09fd211ebb7
I20260320 15:27:30.332949 28733 ts_tablet_manager.cc:1434] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:30.333204 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_30673191803644ec9a6dba7a6ec38dde"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:30.333518 28733 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:30.334049 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30673191803644ec9a6dba7a6ec38dde in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.334410 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_30e46b102a5d409895faee96f35b2b36" } modify_external_catalogs: true
I20260320 15:27:30.334635 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_30e46b102a5d409895faee96f35b2b36" } modify_external_catalogs: true
I20260320 15:27:30.335974 28733 tablet_bootstrap.cc:654] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.336176 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_7744ce92ac4e49be9f107f055594b6b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.336941 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7744ce92ac4e49be9f107f055594b6b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.338588 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_3979e77f6aba4d9f80ba813c92287b84" } new_table_name: "default.Table_0e217a9219f04e1cb94b0c63324773a1" modify_external_catalogs: true
I20260320 15:27:30.341320 28890 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 55d089b5a3c74ca4abb53aeb9c487df4
I20260320 15:27:30.341725 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_9b2e32517f9d49a9a117be78ffddb91e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.342492 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b2e32517f9d49a9a117be78ffddb91e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.345039 28733 tablet_bootstrap.cc:492] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.345271 28733 ts_tablet_manager.cc:1403] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260320 15:27:30.346602 28733 raft_consensus.cc:359] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.346899 28733 raft_consensus.cc:385] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.347029 28733 raft_consensus.cc:740] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.347477 28733 consensus_queue.cc:260] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.348212 28733 ts_tablet_manager.cc:1434] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:30.348242 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:30.348572 28733 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.349433 28902 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_78b650f7f9354416913f3c232c22eb59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.350091 28902 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78b650f7f9354416913f3c232c22eb59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.313136 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.350538 28733 tablet_bootstrap.cc:654] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.350495 28442 raft_consensus.cc:2468] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.351126 28240 leader_election.cc:304] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.351795 29241 raft_consensus.cc:2804] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.351915 29241 raft_consensus.cc:493] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.352038 29241 raft_consensus.cc:3060] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.352530 28734 ts_tablet_manager.cc:1434] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.200s user 0.003s sys 0.000s
I20260320 15:27:30.353328 28447 raft_consensus.cc:2468] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.352779 28444 raft_consensus.cc:2468] T 48fa326348104d148abbba907f4c547f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.354236 28734 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:30.354913 29241 raft_consensus.cc:515] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.356321 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.357918 29241 leader_election.cc:290] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.358145 29246 leader_election.cc:290] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:30.358556 28382 leader_election.cc:343] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.364310 28240 leader_election.cc:304] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.365042 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.365187 29241 raft_consensus.cc:2804] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.365353 29241 raft_consensus.cc:493] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.365444 29241 raft_consensus.cc:3060] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.365549 28444 raft_consensus.cc:3060] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.365643 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.369155 29241 raft_consensus.cc:515] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.370011 28444 raft_consensus.cc:2468] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.370072 29241 leader_election.cc:290] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.371975 28240 leader_election.cc:304] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.372318 29241 raft_consensus.cc:2804] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.372531 29241 raft_consensus.cc:697] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.372619 29446 raft_consensus.cc:493] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.372789 29446 raft_consensus.cc:515] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.373302 29241 consensus_queue.cc:237] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.373492 29446 leader_election.cc:290] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.374377 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.375007 28444 raft_consensus.cc:3060] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.379040 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.379420 28300 raft_consensus.cc:2393] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:30.380348 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.380576 28300 raft_consensus.cc:2468] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.382035 28383 leader_election.cc:304] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.383113 28383 leader_election.cc:304] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.384681 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.384754 29245 raft_consensus.cc:2804] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.385155 29356 consensus_queue.cc:1048] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.387295 28588 raft_consensus.cc:3060] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.391733 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.397174 28733 tablet_bootstrap.cc:492] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.397547 28733 ts_tablet_manager.cc:1403] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.010s
I20260320 15:27:30.398358 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:30.398464 28733 raft_consensus.cc:359] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.398690 28733 raft_consensus.cc:385] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.398782 28733 raft_consensus.cc:740] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.400255 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.401400 29275 consensus_queue.cc:1048] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:30.402208 28239 leader_election.cc:343] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:30.402741 28239 consensus_peers.cc:597] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:30.403149 29275 consensus_queue.cc:1048] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.403252 28733 consensus_queue.cc:260] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.404487 28733 ts_tablet_manager.cc:1434] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260320 15:27:30.406618 28736 tablet_bootstrap.cc:654] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:30.408560 28239 leader_election.cc:343] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.415289 29245 raft_consensus.cc:493] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.415499 29245 raft_consensus.cc:3060] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.416240 29442 raft_consensus.cc:493] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.416469 29442 raft_consensus.cc:515] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.417713 29442 leader_election.cc:290] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.427070 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c2d7e4d26994a17bf2d2615d6552412" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.427000 29446 consensus_queue.cc:1048] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.427352 28447 raft_consensus.cc:2393] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
W20260320 15:27:30.427399 28524 leader_election.cc:343] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.429827 28524 leader_election.cc:304] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:30.430140 29446 raft_consensus.cc:3060] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.426738 29245 raft_consensus.cc:515] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.418112 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.433138 29245 leader_election.cc:290] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.433544 29446 raft_consensus.cc:2749] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.433841 28444 raft_consensus.cc:2468] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.434438 28240 leader_election.cc:304] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.435029 29275 raft_consensus.cc:2804] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.435111 29246 raft_consensus.cc:2749] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.435206 29275 raft_consensus.cc:697] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.435567 29275 consensus_queue.cc:237] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:30.440076 28524 leader_election.cc:343] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.442186 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:30.443378 28382 leader_election.cc:343] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.445003 28734 tablet_bootstrap.cc:654] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.449002 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.450552 28586 raft_consensus.cc:2468] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.451367 29446 raft_consensus.cc:493] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.451576 29446 raft_consensus.cc:515] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.452646 29446 leader_election.cc:290] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.453246 28382 leader_election.cc:304] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.453627 29246 raft_consensus.cc:2804] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.453745 29246 raft_consensus.cc:493] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.453858 29246 raft_consensus.cc:3060] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.461366 28733 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.463786 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:30.463712 29245 consensus_queue.cc:1048] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260320 15:27:30.467227 28733 tablet_bootstrap.cc:654] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.472388 29246 raft_consensus.cc:515] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.475565 29275 consensus_queue.cc:1048] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.475749 28734 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:30.475911 28734 ts_tablet_manager.cc:1403] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.123s user 0.008s sys 0.003s
I20260320 15:27:30.476231 29246 leader_election.cc:290] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.476939 28734 raft_consensus.cc:359] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.481995 28734 raft_consensus.cc:385] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.482479 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:30.482750 28734 raft_consensus.cc:740] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.483392 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:30.484419 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:30.483457 28734 consensus_queue.cc:260] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.485426 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.485630 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.486011 28443 raft_consensus.cc:2393] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:30.485677 28297 raft_consensus.cc:2468] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.486295 28734 ts_tablet_manager.cc:1434] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260320 15:27:30.487646 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.487887 28297 raft_consensus.cc:3060] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.488405 28734 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:30.492027 28297 raft_consensus.cc:2468] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:30.505142 28862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 101855us
I20260320 15:27:30.474701 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.507828 28586 raft_consensus.cc:3060] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.503495 28734 tablet_bootstrap.cc:654] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.479872 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:30.480041 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:30.481077 28557 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:30.481878 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:30.512276 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 889d620c311a42039c22af53ad0802f2 already in progress: creating tablet
I20260320 15:27:30.512239 28586 raft_consensus.cc:2468] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.512950 28382 leader_election.cc:304] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:30.513662 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6231fe30c5b34d0bb7a233abbffeb1e8 already in progress: creating tablet
W20260320 15:27:30.513926 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ee4a0d559ce4dfea878bd3f1c03c09a already in progress: creating tablet
I20260320 15:27:30.513948 29246 raft_consensus.cc:2804] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.514456 29245 raft_consensus.cc:493] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.514248 29246 raft_consensus.cc:697] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:30.514938 29246 consensus_queue.cc:237] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.515323 29245 raft_consensus.cc:515] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.516016 29245 leader_election.cc:290] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.516908 28586 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56de00174db4e2d8b1c7215038a04aa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.517196 28586 raft_consensus.cc:2468] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.517693 28382 leader_election.cc:304] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.518272 29245 raft_consensus.cc:2804] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.518385 29245 raft_consensus.cc:493] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.518501 29245 raft_consensus.cc:3060] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.522547 29245 raft_consensus.cc:515] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.524498 29245 leader_election.cc:290] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.524869 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56de00174db4e2d8b1c7215038a04aa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.525147 28588 raft_consensus.cc:3060] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.529012 28588 raft_consensus.cc:2468] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.529821 28382 leader_election.cc:304] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.531229 29245 raft_consensus.cc:2804] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.531600 29245 raft_consensus.cc:697] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:30.531996 29245 consensus_queue.cc:237] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.533948 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.535044 29485 tablet_replica.cc:333] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:30.535043 28297 raft_consensus.cc:3060] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.535414 29485 raft_consensus.cc:2243] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:30.535784 29485 raft_consensus.cc:2272] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:30.536504 29485 ts_tablet_manager.cc:1916] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:30.540588 28383 leader_election.cc:304] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:30.541236 29442 raft_consensus.cc:2804] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.541427 29442 raft_consensus.cc:697] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:30.541744 29442 consensus_queue.cc:237] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.543778 28733 tablet_bootstrap.cc:492] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.544016 28733 ts_tablet_manager.cc:1403] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.083s user 0.005s sys 0.000s
I20260320 15:27:30.552438 28733 raft_consensus.cc:359] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.552802 28733 raft_consensus.cc:385] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.552948 28733 raft_consensus.cc:740] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.553262 29485 ts_tablet_manager.cc:1929] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:30.553267 28733 consensus_queue.cc:260] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.554168 28733 ts_tablet_manager.cc:1434] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:30.555375 29241 raft_consensus.cc:493] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.555787 28733 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.555709 29241 raft_consensus.cc:515] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.557080 29241 leader_election.cc:290] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.561822 29485 log.cc:1199] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:30.558490 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:30.562302 28733 tablet_bootstrap.cc:654] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.557492 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.567315 28444 raft_consensus.cc:2468] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.568142 29485 ts_tablet_manager.cc:1950] T 825f7e3fdc004fe1a3bcca6deb20cccd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:30.568099 28240 leader_election.cc:304] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.569535 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 825f7e3fdc004fe1a3bcca6deb20cccd (table default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]) successfully deleted
I20260320 15:27:30.571631 29241 raft_consensus.cc:2804] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.571885 29241 raft_consensus.cc:493] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.572158 29241 raft_consensus.cc:3060] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.577594 29430 consensus_queue.cc:1048] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.578042 28733 tablet_bootstrap.cc:492] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.578299 28733 ts_tablet_manager.cc:1403] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.004s
I20260320 15:27:30.579178 28733 raft_consensus.cc:359] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.579478 28733 raft_consensus.cc:385] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.579612 28733 raft_consensus.cc:740] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.580242 28733 consensus_queue.cc:260] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.580850 28733 ts_tablet_manager.cc:1434] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:30.582799 29491 tablet_replica.cc:333] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:30.583117 29491 raft_consensus.cc:2243] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:30.583931 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6231fe30c5b34d0bb7a233abbffeb1e8 already in progress: creating tablet
W20260320 15:27:30.584940 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ee4a0d559ce4dfea878bd3f1c03c09a already in progress: creating tablet
I20260320 15:27:30.591809 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet ad4a30476e7146afae1e77dad078136e
I20260320 15:27:30.596683 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:30.597708 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ad4a30476e7146afae1e77dad078136e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad4a30476e7146afae1e77dad078136e already in progress: creating tablet
I20260320 15:27:30.599165 29246 consensus_queue.cc:1048] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.604744 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5f92561764b4cb9b15bb9a310a350f0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.605651 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56de00174db4e2d8b1c7215038a04aa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.605908 28300 raft_consensus.cc:2468] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.607622 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d56de00174db4e2d8b1c7215038a04aa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.607802 28300 raft_consensus.cc:3060] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:30.610180 28904 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f59a3c9fa17a4babaa9a18f18d51f193 consensus state has no leader
I20260320 15:27:30.610512 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet cea3bf14653f4555acfed1577a79ee4f from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:30.610512 28300 raft_consensus.cc:2468] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.612234 29246 consensus_queue.cc:1048] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.612468 29245 consensus_queue.cc:1048] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.613747 29430 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.613962 28902 master_service.cc:954] ReplaceTablet: received request to replace tablet dd315b5d49104d679b03515b65ee30ac from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:30.613960 29430 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.615118 29430 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.615118 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f38d66983ac842a4bb44792143adeeca" } new_table_name: "default.Table_aff011445f764f9abb08fc4c8d68fc4d" modify_external_catalogs: true
I20260320 15:27:30.574658 28736 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:30.615429 28736 ts_tablet_manager.cc:1403] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.322s user 0.005s sys 0.000s
I20260320 15:27:30.615969 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_602a7434dd7d43a798b06bff668c8d0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:30.616309 28736 raft_consensus.cc:359] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.616618 28736 raft_consensus.cc:385] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:30.616619 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_602a7434dd7d43a798b06bff668c8d0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.616708 28736 raft_consensus.cc:740] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.616961 28736 consensus_queue.cc:260] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.617456 28890 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_e138abbe7dcc46d1babaa051f60268e7" } new_table_name: "default.Table_72b122270f4542a19e5b00a85e754f16" modify_external_catalogs: true
I20260320 15:27:30.618526 28736 ts_tablet_manager.cc:1434] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:30.618919 28901 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_818e1205d4f34148aebe2a4206ca41a5" } new_table_name: "default.Table_a9b80338873049fbaaa6c166ee20099e" modify_external_catalogs: true
W20260320 15:27:30.619027 28240 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:30.618845 28736 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:30.620087 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_32e1ad85cf09464a87ffd6feb70093cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.620692 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32e1ad85cf09464a87ffd6feb70093cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.620755 29430 consensus_queue.cc:1048] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.622251 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.622875 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.644734 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_134b3b5c79e344f2bf710b12615479be" } modify_external_catalogs: true
I20260320 15:27:30.644999 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_134b3b5c79e344f2bf710b12615479be" } modify_external_catalogs: true
I20260320 15:27:30.645944 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 591ccc1d349e4fd599d5e343f91defe2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:30.583508 29491 raft_consensus.cc:2272] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:30.585040 28733 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.611389 28524 leader_election.cc:304] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:30.648094 29445 raft_consensus.cc:2749] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.639694 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:30.650216 28733 tablet_bootstrap.cc:654] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.655449 28734 tablet_bootstrap.cc:492] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:30.639739 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.656427 28734 ts_tablet_manager.cc:1403] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.168s user 0.001s sys 0.004s
I20260320 15:27:30.642807 29449 raft_consensus.cc:493] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.657716 29449 raft_consensus.cc:515] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.658524 29449 leader_election.cc:290] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.657316 28734 raft_consensus.cc:359] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.658810 28734 raft_consensus.cc:385] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.658911 28734 raft_consensus.cc:740] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.659814 29491 ts_tablet_manager.cc:1916] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:30.660176 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.660534 28445 raft_consensus.cc:2393] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:30.661242 28297 raft_consensus.cc:2468] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.662087 28585 raft_consensus.cc:2468] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:30.663592 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_29cefe9b80e84240960c1e41f7401aaa" } new_table_name: "default.Table_7397a25b5cf94613bd1970d6483c2cd0" modify_external_catalogs: true
I20260320 15:27:30.668023 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c081374356e242d6bda444f6aac87050"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.668748 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c081374356e242d6bda444f6aac87050 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.673022 28734 consensus_queue.cc:260] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.680135 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_ea1f04fcc8eb448cac5344c72e695c99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:30.680824 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea1f04fcc8eb448cac5344c72e695c99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.684703 28734 ts_tablet_manager.cc:1434] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
I20260320 15:27:30.686077 28734 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:30.689850 28734 tablet_bootstrap.cc:654] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.694895 29491 ts_tablet_manager.cc:1929] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260320 15:27:30.697013 28890 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76379bf0fa554820be020f647582a72b has no consensus state
I20260320 15:27:30.715860 28734 tablet_bootstrap.cc:492] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:30.716181 28734 ts_tablet_manager.cc:1403] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.002s
I20260320 15:27:30.714003 29449 raft_consensus.cc:493] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.718005 29491 log.cc:1199] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:30.718041 29449 raft_consensus.cc:515] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.718924 29449 leader_election.cc:290] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:30.721763 29491 ts_tablet_manager.cc:1950] T 825f7e3fdc004fe1a3bcca6deb20cccd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:30.722018 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.722338 28445 raft_consensus.cc:2393] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:30.729852 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_108f31bd087a4a56bd7916340d021c75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:30.730979 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_9234513d2cd54f55bf709a1c6fb0de1c" } new_table_name: "default.Table_d6cad22d93234701b9b022843b6fbaaa" modify_external_catalogs: true
I20260320 15:27:30.736478 28734 raft_consensus.cc:359] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.736747 28734 raft_consensus.cc:385] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.736851 28734 raft_consensus.cc:740] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.737125 28734 consensus_queue.cc:260] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.738124 28734 ts_tablet_manager.cc:1434] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260320 15:27:30.749019 28734 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:30.751159 28734 tablet_bootstrap.cc:654] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.752923 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 825f7e3fdc004fe1a3bcca6deb20cccd (table default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]) successfully deleted
W20260320 15:27:30.753312 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 889d620c311a42039c22af53ad0802f2 already in progress: creating tablet
W20260320 15:27:30.753631 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ad4a30476e7146afae1e77dad078136e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad4a30476e7146afae1e77dad078136e already in progress: creating tablet
W20260320 15:27:30.753882 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_108f31bd087a4a56bd7916340d021c75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:30.762979 28733 tablet_bootstrap.cc:492] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.763188 28733 ts_tablet_manager.cc:1403] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.178s user 0.008s sys 0.009s
I20260320 15:27:30.764194 28733 raft_consensus.cc:359] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.764477 28733 raft_consensus.cc:385] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.764614 28733 raft_consensus.cc:740] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.764860 28733 consensus_queue.cc:260] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.765426 28733 ts_tablet_manager.cc:1434] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:30.765780 28733 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.757009 29356 consensus_queue.cc:1048] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.768165 29241 raft_consensus.cc:515] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.768970 29241 leader_election.cc:290] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.770989 28736 tablet_bootstrap.cc:654] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.772207 28733 tablet_bootstrap.cc:654] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.786023 28733 tablet_bootstrap.cc:492] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.786280 28733 ts_tablet_manager.cc:1403] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
W20260320 15:27:30.786485 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 57305.6us
I20260320 15:27:30.787261 28733 raft_consensus.cc:359] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.787325 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.787493 28733 raft_consensus.cc:385] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.787580 28444 raft_consensus.cc:3060] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.787617 28733 raft_consensus.cc:740] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.787889 28733 consensus_queue.cc:260] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:30.788371 28382 consensus_peers.cc:597] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:30.788481 28733 ts_tablet_manager.cc:1434] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:30.788843 28733 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.790225 29449 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.790488 29449 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.791414 29449 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.791867 28444 raft_consensus.cc:2468] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.792301 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.792506 28445 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:30.792668 28240 leader_election.cc:304] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.793023 28524 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.793566 29429 raft_consensus.cc:2804] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.793666 29449 raft_consensus.cc:2804] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.793751 29449 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.793834 29449 raft_consensus.cc:3060] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.793807 29429 raft_consensus.cc:697] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:30.794248 29429 consensus_queue.cc:237] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.797004 29449 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.799407 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:30.799759 28445 raft_consensus.cc:3060] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.802629 28733 tablet_bootstrap.cc:654] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.806258 28445 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:30.806876 28524 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.807202 29445 raft_consensus.cc:2804] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.807353 29445 raft_consensus.cc:697] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:30.807694 29445 consensus_queue.cc:237] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.813791 29449 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:30.816913 28733 tablet_bootstrap.cc:492] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.817099 28733 ts_tablet_manager.cc:1403] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.001s
I20260320 15:27:30.817989 28733 raft_consensus.cc:359] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.818221 28733 raft_consensus.cc:385] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.818331 28733 raft_consensus.cc:740] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.818641 28733 consensus_queue.cc:260] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.819214 28733 ts_tablet_manager.cc:1434] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:30.819578 28733 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.821997 29156 raft_consensus.cc:493] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.822194 29156 raft_consensus.cc:515] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.822281 28733 tablet_bootstrap.cc:654] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.822947 29156 leader_election.cc:290] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.823884 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:30.825385 28382 leader_election.cc:343] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:30.825918 29156 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.826154 29156 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.827092 29156 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.828397 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.832629 28733 tablet_bootstrap.cc:492] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:30.832864 28733 ts_tablet_manager.cc:1403] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260320 15:27:30.833972 28733 raft_consensus.cc:359] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.834208 28733 raft_consensus.cc:385] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:30.834287 28733 raft_consensus.cc:740] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:30.834703 28733 consensus_queue.cc:260] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.835405 28733 ts_tablet_manager.cc:1434] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:30.835855 28733 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:30.838090 28733 tablet_bootstrap.cc:654] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:30.867352 28889 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_93591206ed3f475cb1d454da2ea159a3" } modify_external_catalogs: true
I20260320 15:27:30.868299 28889 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_93591206ed3f475cb1d454da2ea159a3" } modify_external_catalogs: true
I20260320 15:27:30.872100 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b263f9a5df2477e8dc4cd52e5834991" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.872340 28585 raft_consensus.cc:3060] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.875348 28585 raft_consensus.cc:2468] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:30.879890 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb7a910bd2904a40ae63608377206c7e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.880666 29156 raft_consensus.cc:493] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.880834 29156 raft_consensus.cc:515] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.881487 29156 leader_election.cc:290] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.883159 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.883479 28585 raft_consensus.cc:2468] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.884020 28382 leader_election.cc:304] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.884490 29156 raft_consensus.cc:2804] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:30.884660 29156 raft_consensus.cc:493] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:30.884804 29156 raft_consensus.cc:3060] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.890856 29156 raft_consensus.cc:515] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.895326 29156 leader_election.cc:290] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.896139 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:30.896531 28585 raft_consensus.cc:3060] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.900704 28585 raft_consensus.cc:2468] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.901167 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (DEFAULT_TABLE table=default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.901345 28382 leader_election.cc:304] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:30.901733 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008811c90aeb4ca4a4ad5c0190a60dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.902079 29442 raft_consensus.cc:2804] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:30.902630 29442 raft_consensus.cc:697] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:30.902946 29442 consensus_queue.cc:237] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.903924 28425 tablet_service.cc:1511] Processing CreateTablet for tablet c3a49e310b524f3b87eb89f3e84b7800 (DEFAULT_TABLE table=default.Table_536e57665fa14e0e915512c1a27ab8fe [id=f2497b3b3f73471e872d6cba3e653118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.904356 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a49e310b524f3b87eb89f3e84b7800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.904510 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a20d7f22cfa403184e73b24d565b60a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.905742 29511 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.905921 29511 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.909008 28426 tablet_service.cc:1511] Processing CreateTablet for tablet d5095b42b69d40539118c1f63a95f795 (DEFAULT_TABLE table=default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.909535 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5095b42b69d40539118c1f63a95f795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.914031 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (DEFAULT_TABLE table=default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.914556 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008811c90aeb4ca4a4ad5c0190a60dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.915932 28556 tablet_service.cc:1511] Processing CreateTablet for tablet c3a49e310b524f3b87eb89f3e84b7800 (DEFAULT_TABLE table=default.Table_536e57665fa14e0e915512c1a27ab8fe [id=f2497b3b3f73471e872d6cba3e653118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.916038 29445 raft_consensus.cc:493] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:30.916404 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a49e310b524f3b87eb89f3e84b7800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.916275 29445 raft_consensus.cc:515] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:30.919389 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.919961 28561 tablet_service.cc:1511] Processing CreateTablet for tablet d5095b42b69d40539118c1f63a95f795 (DEFAULT_TABLE table=default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.920585 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5095b42b69d40539118c1f63a95f795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.927827 29430 consensus_queue.cc:1048] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.929196 29511 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:30.935812 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:30.935881 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.936141 28444 raft_consensus.cc:2393] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:30.936107 28300 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:30.936647 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.936849 28300 raft_consensus.cc:3060] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.919931 29445 leader_election.cc:290] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:30.940438 28300 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:30.940852 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 8f408aa18d184e5c9a1f900827e70ca8 (DEFAULT_TABLE table=default.Table_ef4a998b01e2437a8edcf95a89e5a314 [id=cf3dff3da9334df6a176702917cfaf18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.941078 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35347116811e4aee8142a503dd21843a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.942360 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 9b838187c21448b6909ad183f9824d2b (DEFAULT_TABLE table=default.Table_602a7434dd7d43a798b06bff668c8d0e [id=cefb97d71bd946b3b75e6d122314bc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.943375 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (DEFAULT_TABLE table=default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.944093 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4a2c8ebc374de9bd8078a757c6be57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.945241 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 8f408aa18d184e5c9a1f900827e70ca8 (DEFAULT_TABLE table=default.Table_ef4a998b01e2437a8edcf95a89e5a314 [id=cf3dff3da9334df6a176702917cfaf18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.945823 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (DEFAULT_TABLE table=default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.946374 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f408aa18d184e5c9a1f900827e70ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.946952 28445 raft_consensus.cc:2393] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:30.947021 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4a2c8ebc374de9bd8078a757c6be57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.948448 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 9b838187c21448b6909ad183f9824d2b (DEFAULT_TABLE table=default.Table_602a7434dd7d43a798b06bff668c8d0e [id=cefb97d71bd946b3b75e6d122314bc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.950467 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 321fe367ec3348f8a049167f3f8c15d5 (DEFAULT_TABLE table=default.Table_5b5bbe0346774e8cbb24c184595d322e [id=c5d8e4f7585a4d72af6d2af8cff2950e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.951081 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321fe367ec3348f8a049167f3f8c15d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.952133 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.952401 28303 raft_consensus.cc:2468] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.952898 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.953140 28303 raft_consensus.cc:2393] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:30.954345 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:30.954569 28303 raft_consensus.cc:2468] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:30.954923 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 321fe367ec3348f8a049167f3f8c15d5 (DEFAULT_TABLE table=default.Table_5b5bbe0346774e8cbb24c184595d322e [id=c5d8e4f7585a4d72af6d2af8cff2950e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.955492 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6f8c20b5904eafb9a62d14bb155c82" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:30.955675 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b838187c21448b6909ad183f9824d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.955698 28303 raft_consensus.cc:3060] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.959138 28303 raft_consensus.cc:2468] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:30.959936 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 49654c67435a48448794879464f6cfb2 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.960516 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49654c67435a48448794879464f6cfb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.966491 28524 leader_election.cc:304] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:30.967262 29445 raft_consensus.cc:3060] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.970965 29445 raft_consensus.cc:2749] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.971752 28524 leader_election.cc:304] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:30.972993 28587 raft_consensus.cc:3060] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.973076 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b838187c21448b6909ad183f9824d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.974694 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f408aa18d184e5c9a1f900827e70ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.976390 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:30.977196 29449 raft_consensus.cc:3060] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.978289 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 49654c67435a48448794879464f6cfb2 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:30.979614 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321fe367ec3348f8a049167f3f8c15d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.979820 29156 consensus_queue.cc:1048] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260320 15:27:30.983772 29449 raft_consensus.cc:2749] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.986323 28524 leader_election.cc:304] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:30.986922 29449 raft_consensus.cc:3060] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:30.988317 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49654c67435a48448794879464f6cfb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:30.990233 29449 raft_consensus.cc:2749] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:30.991505 29156 consensus_queue.cc:1048] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:30.996687 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:30.998495 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:30.999220 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:30.999784 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.000192 28267 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.000921 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (DEFAULT_TABLE table=default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.001410 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008811c90aeb4ca4a4ad5c0190a60dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.005415 29528 tablet_replica.cc:333] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:31.005667 29528 raft_consensus.cc:2243] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:31.006062 29528 raft_consensus.cc:2272] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:31.006686 29528 ts_tablet_manager.cc:1916] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:31.007835 29445 raft_consensus.cc:493] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.008129 29445 raft_consensus.cc:515] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.009212 29445 leader_election.cc:290] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:31.014725 28383 leader_election.cc:304] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:31.022161 28383 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:31.023670 29156 raft_consensus.cc:2804] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:31.024555 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 8f408aa18d184e5c9a1f900827e70ca8 (DEFAULT_TABLE table=default.Table_ef4a998b01e2437a8edcf95a89e5a314 [id=cf3dff3da9334df6a176702917cfaf18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.024989 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f408aa18d184e5c9a1f900827e70ca8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.025058 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 9b838187c21448b6909ad183f9824d2b (DEFAULT_TABLE table=default.Table_602a7434dd7d43a798b06bff668c8d0e [id=cefb97d71bd946b3b75e6d122314bc93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.024501 28268 tablet_service.cc:1511] Processing CreateTablet for tablet d5095b42b69d40539118c1f63a95f795 (DEFAULT_TABLE table=default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.025667 28267 tablet_service.cc:1511] Processing CreateTablet for tablet c3a49e310b524f3b87eb89f3e84b7800 (DEFAULT_TABLE table=default.Table_536e57665fa14e0e915512c1a27ab8fe [id=f2497b3b3f73471e872d6cba3e653118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.025997 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3a49e310b524f3b87eb89f3e84b7800. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.026984 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 321fe367ec3348f8a049167f3f8c15d5 (DEFAULT_TABLE table=default.Table_5b5bbe0346774e8cbb24c184595d322e [id=c5d8e4f7585a4d72af6d2af8cff2950e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.027357 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b838187c21448b6909ad183f9824d2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.027628 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 49654c67435a48448794879464f6cfb2 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.028002 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49654c67435a48448794879464f6cfb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.029820 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (DEFAULT_TABLE table=default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.030241 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f4a2c8ebc374de9bd8078a757c6be57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.031889 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 321fe367ec3348f8a049167f3f8c15d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.026999 28239 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:31.033318 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:31.033668 29511 consensus_queue.cc:1048] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.034092 28239 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.034466 29511 raft_consensus.cc:2749] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.035050 29511 consensus_queue.cc:1048] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.035846 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:31.036409 28904 catalog_manager.cc:6741] ReplaceTablet: tablet cea3bf14653f4555acfed1577a79ee4f deleted and replaced by tablet d04ab44039124f36ac60601098586b4e
I20260320 15:27:31.036580 29444 raft_consensus.cc:3060] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.036934 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:31.037220 29156 raft_consensus.cc:493] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260320 15:27:31.037881 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
I20260320 15:27:31.038972 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:31.040616 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5095b42b69d40539118c1f63a95f795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.041313 29275 consensus_queue.cc:1048] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:31.043211 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
I20260320 15:27:31.043787 29511 consensus_queue.cc:1048] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260320 15:27:31.045024 28239 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.047663 29156 raft_consensus.cc:3060] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.052373 29275 raft_consensus.cc:2749] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:31.049088 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 889d620c311a42039c22af53ad0802f2 already in progress: creating tablet
W20260320 15:27:31.054898 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6231fe30c5b34d0bb7a233abbffeb1e8 already in progress: creating tablet
W20260320 15:27:31.055716 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ee4a0d559ce4dfea878bd3f1c03c09a already in progress: creating tablet
I20260320 15:27:31.063836 28904 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_00ee6904094c4b5abf6e50334f698c40" } modify_external_catalogs: true
I20260320 15:27:31.064056 28904 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_00ee6904094c4b5abf6e50334f698c40" } modify_external_catalogs: true
I20260320 15:27:31.078684 29430 consensus_queue.cc:1048] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.083221 28445 raft_consensus.cc:2468] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:31.088230 29444 raft_consensus.cc:2749] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.090819 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:31.091470 28902 catalog_manager.cc:6741] ReplaceTablet: tablet dd315b5d49104d679b03515b65ee30ac deleted and replaced by tablet ef0d80754dfd4bbd9817f568063511ef
I20260320 15:27:31.092231 29156 raft_consensus.cc:515] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.092393 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:31.091768 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:31.093209 28303 raft_consensus.cc:2468] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:31.093366 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd315b5d49104d679b03515b65ee30ac already in progress: creating tablet
I20260320 15:27:31.096719 29156 leader_election.cc:290] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:31.097340 28524 leader_election.cc:304] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:31.099237 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:31.103792 29445 raft_consensus.cc:2804] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:31.104099 29445 raft_consensus.cc:493] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:31.104298 29445 raft_consensus.cc:3060] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.106936 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:31.108381 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd315b5d49104d679b03515b65ee30ac already in progress: creating tablet
W20260320 15:27:31.108690 28382 leader_election.cc:343] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:31.109130 29445 raft_consensus.cc:515] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.109400 29156 consensus_queue.cc:1048] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.112320 28902 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_fd26120a419a45efa62bf54b58fd74a1" } modify_external_catalogs: true
I20260320 15:27:31.112578 28902 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_fd26120a419a45efa62bf54b58fd74a1" } modify_external_catalogs: true
I20260320 15:27:31.114552 28265 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.115610 29511 consensus_queue.cc:1048] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.118022 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.120687 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:31.120903 28303 raft_consensus.cc:3060] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.123828 29511 consensus_queue.cc:1048] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.124267 28303 raft_consensus.cc:2468] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:31.133845 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:31.134114 28447 raft_consensus.cc:3060] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.136703 29444 consensus_queue.cc:1048] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.137547 28447 raft_consensus.cc:2468] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:31.137795 28383 leader_election.cc:304] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:31.138119 28524 leader_election.cc:304] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:31.138263 29444 raft_consensus.cc:2804] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:31.138396 29444 raft_consensus.cc:697] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:31.138489 29446 raft_consensus.cc:2804] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:31.138629 29446 raft_consensus.cc:697] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:31.138718 29444 consensus_queue.cc:237] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.138897 29446 consensus_queue.cc:237] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.155750 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 197f1935099140a98c5eb801cf44046a
I20260320 15:27:31.156648 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet 6292904782fd4b619483135e58b66684 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:31.158376 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:31.159365 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 197f1935099140a98c5eb801cf44046a because the tablet was not found. No further retry: Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a
I20260320 15:27:31.161926 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:31.164063 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 197f1935099140a98c5eb801cf44046a because the tablet was not found. No further retry: Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a
I20260320 15:27:31.164155 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 57cbefe3a19044f78b64db951df5f7c1 (DEFAULT_TABLE table=default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.164685 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57cbefe3a19044f78b64db951df5f7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.164871 28583 raft_consensus.cc:3060] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.165232 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.179358 28736 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:31.180634 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:31.182350 28902 master_service.cc:954] ReplaceTablet: received request to replace tablet 07ff84dedc074dabb4e8e2d8460066e7 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:31.186257 28561 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:31.187510 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because the tablet was not found. No further retry: Not found: Tablet not found: ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:31.188639 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:31.189456 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because the tablet was not found. No further retry: Not found: Tablet not found: ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:31.195351 29156 consensus_queue.cc:1048] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.199021 29445 leader_election.cc:290] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:31.207248 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 57cbefe3a19044f78b64db951df5f7c1 (DEFAULT_TABLE table=default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.208015 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57cbefe3a19044f78b64db951df5f7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.218766 28734 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:31.204798 29449 consensus_queue.cc:1048] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.224259 29449 raft_consensus.cc:493] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.224455 29449 raft_consensus.cc:515] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.225193 29449 leader_election.cc:290] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:31.205935 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 31379c9e2adc4fd48e0d0b6e9443615a (DEFAULT_TABLE table=default.Table_62a50c277a2f45bd926514ab39b3b7e4 [id=55547a21133840a3a80a02db2c72676d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.225713 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31379c9e2adc4fd48e0d0b6e9443615a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.179714 29511 consensus_queue.cc:1048] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.236320 28736 ts_tablet_manager.cc:1403] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.618s user 0.003s sys 0.001s
I20260320 15:27:31.186058 29275 consensus_queue.cc:1048] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.240859 29275 raft_consensus.cc:493] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.243691 29275 raft_consensus.cc:515] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.248040 28734 ts_tablet_manager.cc:1403] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.499s user 0.014s sys 0.031s
I20260320 15:27:31.250905 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 31379c9e2adc4fd48e0d0b6e9443615a (DEFAULT_TABLE table=default.Table_62a50c277a2f45bd926514ab39b3b7e4 [id=55547a21133840a3a80a02db2c72676d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.254523 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31379c9e2adc4fd48e0d0b6e9443615a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.191717 29528 ts_tablet_manager.cc:1929] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:31.206920 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f70530cd63a0420fb51527759c951e31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:31.208894 28284 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:31.210249 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 57cbefe3a19044f78b64db951df5f7c1 (DEFAULT_TABLE table=default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.269414 28734 raft_consensus.cc:359] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.270726 29446 raft_consensus.cc:493] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.270988 29446 raft_consensus.cc:515] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.271920 29446 leader_election.cc:290] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:31.274119 29511 consensus_queue.cc:1048] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260320 15:27:31.279700 29446 consensus_queue.cc:1048] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.281209 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ea7f260f4edd4fc082ab82880143fa31 (DEFAULT_TABLE table=default.Table_1642210236684e85bbcfdf569f9ae5d6 [id=4a69c667d48e4ae0b73e59fd343e4ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.281848 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7f260f4edd4fc082ab82880143fa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.287022 29528 log.cc:1199] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:31.270368 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:31.277235 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:31.280828 28421 tablet_service.cc:1511] Processing CreateTablet for tablet ea7f260f4edd4fc082ab82880143fa31 (DEFAULT_TABLE table=default.Table_1642210236684e85bbcfdf569f9ae5d6 [id=4a69c667d48e4ae0b73e59fd343e4ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.269903 28734 raft_consensus.cc:385] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:31.290982 29430 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.291235 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57cbefe3a19044f78b64db951df5f7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.303659 28303 raft_consensus.cc:3060] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.303931 29275 leader_election.cc:290] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:31.308563 29528 ts_tablet_manager.cc:1950] T 825f7e3fdc004fe1a3bcca6deb20cccd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:31.310273 29430 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.314505 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 31379c9e2adc4fd48e0d0b6e9443615a (DEFAULT_TABLE table=default.Table_62a50c277a2f45bd926514ab39b3b7e4 [id=55547a21133840a3a80a02db2c72676d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.317435 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31379c9e2adc4fd48e0d0b6e9443615a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.319586 29430 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:31.321131 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 825f7e3fdc004fe1a3bcca6deb20cccd (table default.Table_7879fae3c7d843cbaebc6ccd995af608 [id=3d42fbed387d45c3b87efc83a3d26cfd]) successfully deleted
W20260320 15:27:31.321686 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ad4a30476e7146afae1e77dad078136e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad4a30476e7146afae1e77dad078136e already in progress: creating tablet
I20260320 15:27:31.322664 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b34048a722c3485490637965c60cc5dc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:31.323115 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd315b5d49104d679b03515b65ee30ac already in progress: creating tablet
I20260320 15:27:31.323621 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66a181681cd84b7fb09c3742f4f2222f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:31.324054 28524 leader_election.cc:304] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.326210 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7f260f4edd4fc082ab82880143fa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.324911 28736 raft_consensus.cc:359] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.331240 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:31.331360 28303 raft_consensus.cc:2468] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:31.331897 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:31.332293 28301 raft_consensus.cc:2393] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:31.332770 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:31.332962 28734 raft_consensus.cc:740] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:31.333320 28900 catalog_manager.cc:5671] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } }
I20260320 15:27:31.334581 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 325caab9b8f34f04a97b41137652ccdb (DEFAULT_TABLE table=default.Table_047ea6293f154736b2bff4875bc8cd43 [id=4685dae9caa842d6bc5fe1c294a37af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.339217 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325caab9b8f34f04a97b41137652ccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.335346 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:31.342226 28734 consensus_queue.cc:260] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.352524 28734 ts_tablet_manager.cc:1434] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.104s user 0.001s sys 0.001s
I20260320 15:27:31.353130 29442 consensus_queue.cc:1048] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260320 15:27:31.354732 28282 tablet_service.cc:1511] Processing CreateTablet for tablet ea7f260f4edd4fc082ab82880143fa31 (DEFAULT_TABLE table=default.Table_1642210236684e85bbcfdf569f9ae5d6 [id=4a69c667d48e4ae0b73e59fd343e4ff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.355342 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea7f260f4edd4fc082ab82880143fa31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.356448 28267 tablet_service.cc:1511] Processing CreateTablet for tablet 325caab9b8f34f04a97b41137652ccdb (DEFAULT_TABLE table=default.Table_047ea6293f154736b2bff4875bc8cd43 [id=4685dae9caa842d6bc5fe1c294a37af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.347265 28587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:31.357057 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325caab9b8f34f04a97b41137652ccdb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.357991 28240 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
W20260320 15:27:31.358878 28240 leader_election.cc:343] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:31.360998 28524 leader_election.cc:304] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.361681 29546 raft_consensus.cc:3060] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:31.362174 28736 raft_consensus.cc:385] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:31.362673 28736 raft_consensus.cc:740] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:31.363212 28736 consensus_queue.cc:260] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.364603 28736 ts_tablet_manager.cc:1434] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.128s user 0.002s sys 0.002s
I20260320 15:27:31.364921 29430 consensus_queue.cc:1048] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.371965 29547 raft_consensus.cc:2749] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:31.375945 28239 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:31.376152 28239 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:31.376809 28239 leader_election.cc:343] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:31.376972 28239 leader_election.cc:304] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.377262 29512 consensus_queue.cc:1048] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.377807 29512 raft_consensus.cc:2749] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.378039 29512 raft_consensus.cc:2749] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.378628 29511 consensus_queue.cc:1048] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.380235 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:31.380666 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 78288f7aea204870af75060476760ee5 (DEFAULT_TABLE table=default.Table_57819bbc59d24aa1a4d2a28ac5e50728 [id=f74f702b87f24a15bcdca554d3a0bbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.381327 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78288f7aea204870af75060476760ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.385917 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 325caab9b8f34f04a97b41137652ccdb (DEFAULT_TABLE table=default.Table_047ea6293f154736b2bff4875bc8cd43 [id=4685dae9caa842d6bc5fe1c294a37af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.388056 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 008621b6a3fc40149a1c6aff8b12ef4e (DEFAULT_TABLE table=default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 [id=462e3845a6664e418282ec357ae39ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.388734 29430 consensus_queue.cc:1048] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.402287 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008621b6a3fc40149a1c6aff8b12ef4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.403595 28733 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.564s user 0.087s sys 0.148s
W20260320 15:27:31.403754 28733 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.565s user 0.087s sys 0.148s
I20260320 15:27:31.403788 29430 consensus_queue.cc:1048] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.406080 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 325caab9b8f34f04a97b41137652ccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.407938 28733 tablet_bootstrap.cc:492] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:31.409662 29546 raft_consensus.cc:2749] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.420535 28733 ts_tablet_manager.cc:1403] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.580s user 0.088s sys 0.149s
I20260320 15:27:31.427970 28733 raft_consensus.cc:359] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.428241 28733 raft_consensus.cc:385] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:31.428589 28733 raft_consensus.cc:740] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:31.428861 28733 consensus_queue.cc:260] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.441196 28733 ts_tablet_manager.cc:1434] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260320 15:27:31.441639 28733 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:31.443331 29444 raft_consensus.cc:493] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.455401 29245 consensus_queue.cc:1048] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260320 15:27:31.458726 29356 consensus_queue.cc:1048] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.468498 28733 tablet_bootstrap.cc:654] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:31.483119 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:31.489764 29156 consensus_queue.cc:1048] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.497920 29509 raft_consensus.cc:493] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:31.498118 29509 raft_consensus.cc:515] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.498893 29509 leader_election.cc:290] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:31.499666 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:31.500306 28239 leader_election.cc:343] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:31.539929 29444 raft_consensus.cc:515] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.540876 29444 leader_election.cc:290] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:31.541046 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:31.543350 28584 raft_consensus.cc:3060] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:31.543634 28239 consensus_peers.cc:597] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:31.544103 28383 leader_election.cc:343] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b395977eb0444bcf8563cd8774fab40f
W20260320 15:27:31.548751 28382 leader_election.cc:343] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:31.548990 28382 leader_election.cc:304] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:31.549602 29444 raft_consensus.cc:2749] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:31.555523 28240 leader_election.cc:343] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:31.555828 28240 leader_election.cc:304] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:31.556445 29555 raft_consensus.cc:2749] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:31.558429 28734 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:31.565054 29423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260320 15:27:31.565747 29423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260320 15:27:31.565999 29423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260320 15:27:31.573115 28736 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:31.576220 28736 tablet_bootstrap.cc:654] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:31.584383 29423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260320 15:27:31.584668 29423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260320 15:27:31.584851 29423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260320 15:27:31.584990 29423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260320 15:27:31.598412 28734 tablet_bootstrap.cc:654] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:31.607292 29555 consensus_queue.cc:1048] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:31.511049 28862 consensus_peers.cc:597] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 -> Peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Couldn't send request to peer 07e8bb8819eb4cb4892e0cba1f758892. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260320 15:27:31.613343 28862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 102270us
W20260320 15:27:31.343132 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
W20260320 15:27:31.614105 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 271002us
W20260320 15:27:31.615443 28873 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
I20260320 15:27:31.615949 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 008621b6a3fc40149a1c6aff8b12ef4e (DEFAULT_TABLE table=default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 [id=462e3845a6664e418282ec357ae39ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.616482 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008621b6a3fc40149a1c6aff8b12ef4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.616062 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 197f1935099140a98c5eb801cf44046a because the tablet was not found. No further retry: Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a
I20260320 15:27:31.617117 29423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.74.20:43127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.61:44739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.74.61
--webserver_port=39103
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 68f10dd41067059260a5fb5c725b178b69c38aed
build type FASTDEBUG
built by None at 20 Mar 2026 14:43:41 UTC on bdcb31816ec0
build id 11023
ASAN enabled
W20260320 15:27:31.617916 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because the tablet was not found. No further retry: Not found: Tablet not found: ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:31.618939 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 008621b6a3fc40149a1c6aff8b12ef4e (DEFAULT_TABLE table=default.Table_7a7fc87f01474fd899f44ff7a0c1e8a4 [id=462e3845a6664e418282ec357ae39ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.619694 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 008621b6a3fc40149a1c6aff8b12ef4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.620291 29423 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260320 15:27:31.623318 29423 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260320 15:27:31.632675 29444 consensus_queue.cc:1048] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.633230 29581 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:31.633419 29581 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:31.635277 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 6 candidate_status { last_received { term: 5 index: 582 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:31.640805 29581 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
I20260320 15:27:31.642935 28561 tablet_service.cc:1511] Processing CreateTablet for tablet 11bef661385b4221a991710ffb34dccb (DEFAULT_TABLE table=default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 [id=2a94c97148eb4bdf8abb574cb9d30204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.643496 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bef661385b4221a991710ffb34dccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.645941 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 (DEFAULT_TABLE table=default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.646556 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e9791ac61b34fc09a8292c79f0c4032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.648768 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 78288f7aea204870af75060476760ee5 (DEFAULT_TABLE table=default.Table_57819bbc59d24aa1a4d2a28ac5e50728 [id=f74f702b87f24a15bcdca554d3a0bbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.649250 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78288f7aea204870af75060476760ee5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.658877 28109 leader_election.cc:336] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:31.659224 28109 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:31.659648 29581 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260320 15:27:31.659709 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 78288f7aea204870af75060476760ee5 (DEFAULT_TABLE table=default.Table_57819bbc59d24aa1a4d2a28ac5e50728 [id=f74f702b87f24a15bcdca554d3a0bbdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.660979 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78288f7aea204870af75060476760ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.664126 29156 consensus_queue.cc:1048] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:31.664803 29585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:31.665860 28425 tablet_service.cc:1511] Processing CreateTablet for tablet b1b6529e062440828fca2e18b129f8e3 (DEFAULT_TABLE table=default.Table_294d5eabb8df40cd888a0769290746bb [id=0350c3a65a5e449dbbdb140b2d82d60a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.666102 28265 tablet_service.cc:1511] Processing CreateTablet for tablet b1b6529e062440828fca2e18b129f8e3 (DEFAULT_TABLE table=default.Table_294d5eabb8df40cd888a0769290746bb [id=0350c3a65a5e449dbbdb140b2d82d60a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.666525 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b6529e062440828fca2e18b129f8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.666551 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b6529e062440828fca2e18b129f8e3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.668629 29586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:31.673031 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 11bef661385b4221a991710ffb34dccb (DEFAULT_TABLE table=default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 [id=2a94c97148eb4bdf8abb574cb9d30204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.673765 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bef661385b4221a991710ffb34dccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.678395 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:31.681154 29423 server_base.cc:1061] running on GCE node
W20260320 15:27:31.681854 29590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260320 15:27:31.683228 29423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260320 15:27:31.688697 29423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260320 15:27:31.692214 29423 hybrid_clock.cc:648] HybridClock initialized: now 1774020451692067 us; error 135 us; skew 500 ppm
I20260320 15:27:31.692996 29423 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260320 15:27:31.704355 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 (DEFAULT_TABLE table=default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.704958 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e9791ac61b34fc09a8292c79f0c4032. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:31.719709 29560 log.cc:927] Time spent T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Append to log took a long time: real 0.179s user 0.000s sys 0.000s
I20260320 15:27:31.732079 28736 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:31.732628 28736 ts_tablet_manager.cc:1403] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.161s user 0.004s sys 0.001s
I20260320 15:27:31.733920 28736 raft_consensus.cc:359] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:31.734248 28265 tablet_service.cc:1511] Processing CreateTablet for tablet 11bef661385b4221a991710ffb34dccb (DEFAULT_TABLE table=default.Table_602ab71d0a9a4a3cb69401e8e2b9ffe9 [id=2a94c97148eb4bdf8abb574cb9d30204]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.734501 28736 raft_consensus.cc:385] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:31.734730 28736 raft_consensus.cc:740] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:31.734889 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bef661385b4221a991710ffb34dccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.757793 28559 tablet_service.cc:1511] Processing CreateTablet for tablet b1b6529e062440828fca2e18b129f8e3 (DEFAULT_TABLE table=default.Table_294d5eabb8df40cd888a0769290746bb [id=0350c3a65a5e449dbbdb140b2d82d60a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.758302 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b6529e062440828fca2e18b129f8e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.765499 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 20eec704800e4d8b924d97556cc2700e (DEFAULT_TABLE table=default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.767206 28733 tablet_bootstrap.cc:492] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:31.767673 28556 tablet_service.cc:1511] Processing CreateTablet for tablet f6240d441c624b238f7c13eb3c5fa9fc (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.771392 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 7aba58642f7049b1aa0536b6ab6cb271 (DEFAULT_TABLE table=default.Table_d643a0220e4847efb0a5e33127808a91 [id=c759d0cd0eb64c76812102f626211d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.772505 29246 consensus_queue.cc:1048] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260320 15:27:31.779036 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfbc7338eda489987315e681ee44146 (DEFAULT_TABLE table=default.Table_4a66f1fda9e9438aad057009144db449 [id=20d83b20838641f0ae67bcd3653a0ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.779268 28267 tablet_service.cc:1511] Processing CreateTablet for tablet f6240d441c624b238f7c13eb3c5fa9fc (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.779676 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfbc7338eda489987315e681ee44146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.779961 28282 tablet_service.cc:1511] Processing CreateTablet for tablet 20eec704800e4d8b924d97556cc2700e (DEFAULT_TABLE table=default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.780385 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20eec704800e4d8b924d97556cc2700e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.782200 28278 tablet_service.cc:1511] Processing CreateTablet for tablet e2243e6811744b7cba99a487f8e8bb83 (DEFAULT_TABLE table=default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.782786 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2243e6811744b7cba99a487f8e8bb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.783946 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 54c75b9deab44c829f6720480e38c80b (DEFAULT_TABLE table=default.Table_fbb25a9824fc45c7b85abec85ccb75b8 [id=14aef9640e7d427ba93c8b21d7c332c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.784099 28272 tablet_service.cc:1511] Processing CreateTablet for tablet bcf4e017e2e743d7b38628d6f1e0d72e (DEFAULT_TABLE table=default.Table_eb8fe7340adc409c98903887ac581604 [id=3b4245a4113946c2838ad27642634e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.784449 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c75b9deab44c829f6720480e38c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.784873 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 7aba58642f7049b1aa0536b6ab6cb271 (DEFAULT_TABLE table=default.Table_d643a0220e4847efb0a5e33127808a91 [id=c759d0cd0eb64c76812102f626211d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.785265 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aba58642f7049b1aa0536b6ab6cb271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.788017 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 4150c4c26a26405b9be16f66d15a8948 (DEFAULT_TABLE table=default.Table_12077503f4ba430396cb7392d0e64fa7 [id=18d21a3e06bb4a3f9bb6416cd95047e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.788166 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 594357400f8c44138c7c4f11bf6d5afd (DEFAULT_TABLE table=default.Table_b3b4c26f9c844ac9a86b26c019bd713e [id=514f5d793c564cd9b7fd8d1fc84970f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.788606 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150c4c26a26405b9be16f66d15a8948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.789903 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf4e017e2e743d7b38628d6f1e0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.793462 28273 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb7cbe0a9224412880144927ec8fa99 (DEFAULT_TABLE table=default.Table_a5cb91f5ca204475836edc1cb0ee1035 [id=194ce4979c0d44ce81f03c1764a09c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.795751 28267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6240d441c624b238f7c13eb3c5fa9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.796507 28273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb7cbe0a9224412880144927ec8fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.796794 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594357400f8c44138c7c4f11bf6d5afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.797459 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:31.798084 28900 catalog_manager.cc:5671] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } }
I20260320 15:27:31.798506 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:31.798776 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:31.799289 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:31.800455 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:31.800822 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:31.801113 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:31.801404 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:31.801890 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:31.803700 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]). Expected version 1 got 0
I20260320 15:27:31.804075 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:31.804479 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:31.804708 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 547d82cee19b4959b3a779c8b8088e83 (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.805011 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 1 got 0
I20260320 15:27:31.805207 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d82cee19b4959b3a779c8b8088e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.805338 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:31.805576 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:31.805847 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:31.812842 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:31.813445 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 7aba58642f7049b1aa0536b6ab6cb271 (DEFAULT_TABLE table=default.Table_d643a0220e4847efb0a5e33127808a91 [id=c759d0cd0eb64c76812102f626211d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.813993 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aba58642f7049b1aa0536b6ab6cb271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.814384 28275 tablet_service.cc:1511] Processing CreateTablet for tablet 197f1935099140a98c5eb801cf44046a (DEFAULT_TABLE table=default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.818373 28421 tablet_service.cc:1511] Processing CreateTablet for tablet f6240d441c624b238f7c13eb3c5fa9fc (DEFAULT_TABLE table=default.Table_15707876735c420fb2d7fbc028f9b1d0 [id=cf9c764d6b874bd1861111099488895e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.819566 28419 tablet_service.cc:1511] Processing CreateTablet for tablet 594357400f8c44138c7c4f11bf6d5afd (DEFAULT_TABLE table=default.Table_b3b4c26f9c844ac9a86b26c019bd713e [id=514f5d793c564cd9b7fd8d1fc84970f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.820312 28425 tablet_service.cc:1511] Processing CreateTablet for tablet bcf4e017e2e743d7b38628d6f1e0d72e (DEFAULT_TABLE table=default.Table_eb8fe7340adc409c98903887ac581604 [id=3b4245a4113946c2838ad27642634e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.820186 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 (DEFAULT_TABLE table=default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.820571 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 20eec704800e4d8b924d97556cc2700e (DEFAULT_TABLE table=default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.821173 28269 tablet_service.cc:1511] Processing CreateTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.821389 28279 tablet_service.cc:1511] Processing CreateTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 (DEFAULT_TABLE table=default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.822149 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 4190329df1b644c392ff8cbace9aae7a (DEFAULT_TABLE table=default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.822310 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:31.822573 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:31.822726 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 1 got 0
I20260320 15:27:31.822974 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:31.823351 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:31.823572 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:31.823747 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:31.824040 28900 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:31.824223 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 514e7b02a28142c2b371644089044a9d (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.825098 28275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197f1935099140a98c5eb801cf44046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.825539 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e9791ac61b34fc09a8292c79f0c4032. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.825824 28269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c1002cbdd845e2b96dcb57c450cc6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.826568 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514e7b02a28142c2b371644089044a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.828631 28283 tablet_service.cc:1511] Processing CreateTablet for tablet da29d2ffcb07428789c22a573d680289 (DEFAULT_TABLE table=default.Table_0f2d272a74ac4a8ab851d662826ba9e6 [id=611935c42b0e42c7b8b6fa719c526910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.830804 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.833086 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4190329df1b644c392ff8cbace9aae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.834702 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 54c75b9deab44c829f6720480e38c80b (DEFAULT_TABLE table=default.Table_fbb25a9824fc45c7b85abec85ccb75b8 [id=14aef9640e7d427ba93c8b21d7c332c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.843753 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 4190329df1b644c392ff8cbace9aae7a (DEFAULT_TABLE table=default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.847275 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed5d220ec4b406aa5bae970f785de63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.847818 28279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3422cf33324ab89a2b44e27cae0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.848515 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da29d2ffcb07428789c22a573d680289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.850669 28412 tablet_service.cc:1511] Processing CreateTablet for tablet 4150c4c26a26405b9be16f66d15a8948 (DEFAULT_TABLE table=default.Table_12077503f4ba430396cb7392d0e64fa7 [id=18d21a3e06bb4a3f9bb6416cd95047e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.853715 28408 tablet_service.cc:1511] Processing CreateTablet for tablet b20efd94b09746c88bb48ba36af4759e (DEFAULT_TABLE table=default.Table_103c6e9e031144a1bc44f351aa73d587 [id=6a055fea11de4e39addc011ed3a3fe4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.854553 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 197f1935099140a98c5eb801cf44046a (DEFAULT_TABLE table=default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.860908 28410 tablet_service.cc:1511] Processing CreateTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.862409 28418 tablet_service.cc:1511] Processing CreateTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 (DEFAULT_TABLE table=default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.864344 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 514e7b02a28142c2b371644089044a9d (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.923916 28280 tablet_service.cc:1511] Processing CreateTablet for tablet b20efd94b09746c88bb48ba36af4759e (DEFAULT_TABLE table=default.Table_103c6e9e031144a1bc44f351aa73d587 [id=6a055fea11de4e39addc011ed3a3fe4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.929134 28274 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2f5513684da79f6eebbed89d0392 (DEFAULT_TABLE table=default.Table_a2607c8356f048f1bb3fbe1f240848c2 [id=713c4ac167384536b97796e4ba84e04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.929672 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20efd94b09746c88bb48ba36af4759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.931417 29553 consensus_queue.cc:1048] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:31.931633 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2f5513684da79f6eebbed89d0392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.934651 28271 tablet_service.cc:1511] Processing CreateTablet for tablet eb011d672f244585b90272443897ed5a (DEFAULT_TABLE table=default.Table_186af3c003da48d78f6c983d6f7b63d8 [id=8f5be23c174e487dacf40ad504ead589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.935103 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb011d672f244585b90272443897ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.958573 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb3d5e19cda48fda124b6238902c03a (DEFAULT_TABLE table=default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.959715 28414 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb7cbe0a9224412880144927ec8fa99 (DEFAULT_TABLE table=default.Table_a5cb91f5ca204475836edc1cb0ee1035 [id=194ce4979c0d44ce81f03c1764a09c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.961045 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 233f6a8e9369411b8e0c35a167c55ff6 (DEFAULT_TABLE table=default.Table_7b794017a8044d3a82313842a3da9995 [id=792a35a32922494599ee94725386a2a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.962604 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb3d5e19cda48fda124b6238902c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.979228 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 813604f6c6c747eda18f7a01d9d8ec97 (DEFAULT_TABLE table=default.Table_65a8a44c03cd4385a65c2e752446c84c [id=81f681463e614a42b8730cace88a123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.980079 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233f6a8e9369411b8e0c35a167c55ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.983561 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813604f6c6c747eda18f7a01d9d8ec97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.984608 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20eec704800e4d8b924d97556cc2700e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.984987 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6240d441c624b238f7c13eb3c5fa9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.987620 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 594357400f8c44138c7c4f11bf6d5afd (DEFAULT_TABLE table=default.Table_b3b4c26f9c844ac9a86b26c019bd713e [id=514f5d793c564cd9b7fd8d1fc84970f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.988144 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594357400f8c44138c7c4f11bf6d5afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:31.995205 28550 tablet_service.cc:1511] Processing CreateTablet for tablet bcf4e017e2e743d7b38628d6f1e0d72e (DEFAULT_TABLE table=default.Table_eb8fe7340adc409c98903887ac581604 [id=3b4245a4113946c2838ad27642634e80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.994892 28549 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfbc7338eda489987315e681ee44146 (DEFAULT_TABLE table=default.Table_4a66f1fda9e9438aad057009144db449 [id=20d83b20838641f0ae67bcd3653a0ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:31.998560 28733 ts_tablet_manager.cc:1403] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.326s user 0.039s sys 0.068s
I20260320 15:27:31.998711 29445 consensus_queue.cc:1048] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260320 15:27:32.004271 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 54c75b9deab44c829f6720480e38c80b (DEFAULT_TABLE table=default.Table_fbb25a9824fc45c7b85abec85ccb75b8 [id=14aef9640e7d427ba93c8b21d7c332c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.004207 28565 tablet_service.cc:1511] Processing CreateTablet for tablet e2243e6811744b7cba99a487f8e8bb83 (DEFAULT_TABLE table=default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.006469 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 4150c4c26a26405b9be16f66d15a8948 (DEFAULT_TABLE table=default.Table_12077503f4ba430396cb7392d0e64fa7 [id=18d21a3e06bb4a3f9bb6416cd95047e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.007404 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf4e017e2e743d7b38628d6f1e0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.009734 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 547d82cee19b4959b3a779c8b8088e83 (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.009281 29446 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s user 0.046s sys 0.115s
I20260320 15:27:32.011679 29547 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.013100 29547 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.014945 29547 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:32.016562 28733 raft_consensus.cc:359] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.017180 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb7cbe0a9224412880144927ec8fa99 (DEFAULT_TABLE table=default.Table_a5cb91f5ca204475836edc1cb0ee1035 [id=194ce4979c0d44ce81f03c1764a09c78]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.028290 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:32.055449 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 4190329df1b644c392ff8cbace9aae7a (DEFAULT_TABLE table=default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.055931 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2243e6811744b7cba99a487f8e8bb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.056368 28568 tablet_service.cc:1511] Processing CreateTablet for tablet da29d2ffcb07428789c22a573d680289 (DEFAULT_TABLE table=default.Table_0f2d272a74ac4a8ab851d662826ba9e6 [id=611935c42b0e42c7b8b6fa719c526910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.056993 28566 tablet_service.cc:1511] Processing CreateTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.057472 29445 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.057495 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da29d2ffcb07428789c22a573d680289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.057838 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150c4c26a26405b9be16f66d15a8948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.058032 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c75b9deab44c829f6720480e38c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.059407 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d82cee19b4959b3a779c8b8088e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.060228 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfbc7338eda489987315e681ee44146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.061379 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c1002cbdd845e2b96dcb57c450cc6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.061791 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4190329df1b644c392ff8cbace9aae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.062052 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aba58642f7049b1aa0536b6ab6cb271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.061992 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 197f1935099140a98c5eb801cf44046a (DEFAULT_TABLE table=default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.062242 28733 raft_consensus.cc:385] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.062283 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 514e7b02a28142c2b371644089044a9d (DEFAULT_TABLE table=default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.062372 28733 raft_consensus.cc:740] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.062476 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197f1935099140a98c5eb801cf44046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.062670 28733 consensus_queue.cc:260] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.062676 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514e7b02a28142c2b371644089044a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.063195 28733 ts_tablet_manager.cc:1434] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.064s user 0.001s sys 0.001s
I20260320 15:27:32.062749 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 (DEFAULT_TABLE table=default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.065758 28733 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:32.075628 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb7cbe0a9224412880144927ec8fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.082417 28736 consensus_queue.cc:260] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:32.088882 29446 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s user 0.046s sys 0.116s
I20260320 15:27:32.090047 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3422cf33324ab89a2b44e27cae0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.096824 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68 (DEFAULT_TABLE table=default.Table_fda3998f623540e6beca0a6f3455b0b3 [id=9e38178c12314306a0b595904a402f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.097412 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.098191 28736 ts_tablet_manager.cc:1434] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.365s user 0.001s sys 0.001s
I20260320 15:27:32.100562 28280 tablet_service.cc:1511] Processing CreateTablet for tablet db16d86664794305bde714a7ff4d5caf (DEFAULT_TABLE table=default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.101197 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db16d86664794305bde714a7ff4d5caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.100950 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.101926 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bbc5d5f49a04216a352ddb1b571dce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.103411 29423 webserver.cc:492] Webserver started at http://127.27.74.61:39103/ using document root <none> and password file <none>
I20260320 15:27:32.104460 28270 tablet_service.cc:1511] Processing CreateTablet for tablet 26ae379304674bc3941af125f0d9d6b3 (DEFAULT_TABLE table=default.Table_ce843e88d7f24053b0be457f201e11ae [id=9941875dffce49d680808963e123cdbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.105020 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ae379304674bc3941af125f0d9d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.109977 29423 fs_manager.cc:362] Metadata directory not provided
I20260320 15:27:32.112383 29423 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260320 15:27:32.116837 29547 consensus_queue.cc:1048] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260320 15:27:32.057705 29445 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.120232 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 76379bf0fa554820be020f647582a72b (DEFAULT_TABLE table=default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.120847 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76379bf0fa554820be020f647582a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.125679 28736 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:32.126278 29445 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:32.151428 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:32.154071 28558 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2f5513684da79f6eebbed89d0392 (DEFAULT_TABLE table=default.Table_a2607c8356f048f1bb3fbe1f240848c2 [id=713c4ac167384536b97796e4ba84e04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.163129 28564 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.163128 28561 tablet_service.cc:1511] Processing CreateTablet for tablet b20efd94b09746c88bb48ba36af4759e (DEFAULT_TABLE table=default.Table_103c6e9e031144a1bc44f351aa73d587 [id=6a055fea11de4e39addc011ed3a3fe4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.166018 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2f5513684da79f6eebbed89d0392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.167126 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd1799035a3431092fbdfad3c42d22b (DEFAULT_TABLE table=default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.167894 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd1799035a3431092fbdfad3c42d22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.179139 28561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20efd94b09746c88bb48ba36af4759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.184402 28564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed5d220ec4b406aa5bae970f785de63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.184507 28552 tablet_service.cc:1511] Processing CreateTablet for tablet eb011d672f244585b90272443897ed5a (DEFAULT_TABLE table=default.Table_186af3c003da48d78f6c983d6f7b63d8 [id=8f5be23c174e487dacf40ad504ead589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.194379 28411 tablet_service.cc:1511] Processing CreateTablet for tablet e2243e6811744b7cba99a487f8e8bb83 (DEFAULT_TABLE table=default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.195353 29599 raft_consensus.cc:493] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.212647 28733 tablet_bootstrap.cc:654] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.217149 29599 raft_consensus.cc:515] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.222764 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 547d82cee19b4959b3a779c8b8088e83 (DEFAULT_TABLE table=default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.223479 28417 tablet_service.cc:1511] Processing CreateTablet for tablet da29d2ffcb07428789c22a573d680289 (DEFAULT_TABLE table=default.Table_0f2d272a74ac4a8ab851d662826ba9e6 [id=611935c42b0e42c7b8b6fa719c526910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.223997 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.223515 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6240d441c624b238f7c13eb3c5fa9fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.233086 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:32.233413 28298 raft_consensus.cc:2393] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:32.236312 29599 leader_election.cc:290] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:32.238891 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da29d2ffcb07428789c22a573d680289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.243026 29444 consensus_queue.cc:1048] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.055s
I20260320 15:27:32.245388 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed5d220ec4b406aa5bae970f785de63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:32.247649 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 241915us
I20260320 15:27:32.248132 29156 consensus_queue.cc:1048] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:32.252681 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfbc7338eda489987315e681ee44146 (DEFAULT_TABLE table=default.Table_4a66f1fda9e9438aad057009144db449 [id=20d83b20838641f0ae67bcd3653a0ef4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.254050 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 547d82cee19b4959b3a779c8b8088e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.268359 28549 tablet_service.cc:1511] Processing CreateTablet for tablet 233f6a8e9369411b8e0c35a167c55ff6 (DEFAULT_TABLE table=default.Table_7b794017a8044d3a82313842a3da9995 [id=792a35a32922494599ee94725386a2a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.284134 28734 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.645s user 0.037s sys 0.130s
W20260320 15:27:32.287668 28423 consensus_meta.cc:287] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: Time spent flushing consensus metadata: real 0.616s user 0.005s sys 0.000s
I20260320 15:27:32.300318 28736 tablet_bootstrap.cc:700] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Time spent opening tablet: real 0.170s user 0.000s sys 0.000s
I20260320 15:27:32.300925 28736 tablet_bootstrap.cc:654] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.302963 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb011d672f244585b90272443897ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.281796 28554 tablet_service.cc:1511] Processing CreateTablet for tablet 813604f6c6c747eda18f7a01d9d8ec97 (DEFAULT_TABLE table=default.Table_65a8a44c03cd4385a65c2e752446c84c [id=81f681463e614a42b8730cace88a123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.316339 28557 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb3d5e19cda48fda124b6238902c03a (DEFAULT_TABLE table=default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.324820 28734 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.685s user 0.037s sys 0.131s
I20260320 15:27:32.328241 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89425da569794a8fa972817023bf853d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:32.328918 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 26ae379304674bc3941af125f0d9d6b3 (DEFAULT_TABLE table=default.Table_ce843e88d7f24053b0be457f201e11ae [id=9941875dffce49d680808963e123cdbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.331103 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2243e6811744b7cba99a487f8e8bb83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.341609 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 76379bf0fa554820be020f647582a72b (DEFAULT_TABLE table=default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.345705 28568 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd1799035a3431092fbdfad3c42d22b (DEFAULT_TABLE table=default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.354274 28424 consensus_meta.cc:287] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: Time spent flushing consensus metadata: real 0.527s user 0.007s sys 0.002s
I20260320 15:27:32.355160 28424 tablet_service.cc:1511] Processing CreateTablet for tablet eb011d672f244585b90272443897ed5a (DEFAULT_TABLE table=default.Table_186af3c003da48d78f6c983d6f7b63d8 [id=8f5be23c174e487dacf40ad504ead589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.361730 28423 tablet_service.cc:1511] Processing CreateTablet for tablet d87a2f5513684da79f6eebbed89d0392 (DEFAULT_TABLE table=default.Table_a2607c8356f048f1bb3fbe1f240848c2 [id=713c4ac167384536b97796e4ba84e04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.362653 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20eec704800e4d8b924d97556cc2700e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.375313 28554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813604f6c6c747eda18f7a01d9d8ec97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.380230 29599 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.382448 28734 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:32.382661 28734 ts_tablet_manager.cc:1403] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 1.030s user 0.038s sys 0.134s
I20260320 15:27:32.387660 28734 raft_consensus.cc:359] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.387996 28734 raft_consensus.cc:385] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.388911 28734 raft_consensus.cc:740] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.389225 28734 consensus_queue.cc:260] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.390538 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb3d5e19cda48fda124b6238902c03a (DEFAULT_TABLE table=default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.391183 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76379bf0fa554820be020f647582a72b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.391247 28734 ts_tablet_manager.cc:1434] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260320 15:27:32.394250 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d87a2f5513684da79f6eebbed89d0392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.394193 29600 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.401469 28549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233f6a8e9369411b8e0c35a167c55ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.403067 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb011d672f244585b90272443897ed5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.406040 28415 tablet_service.cc:1511] Processing CreateTablet for tablet 233f6a8e9369411b8e0c35a167c55ff6 (DEFAULT_TABLE table=default.Table_7b794017a8044d3a82313842a3da9995 [id=792a35a32922494599ee94725386a2a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:32.408651 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:32.408993 29599 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.409219 28734 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:32.410530 28565 tablet_service.cc:1511] Processing CreateTablet for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68 (DEFAULT_TABLE table=default.Table_fda3998f623540e6beca0a6f3455b0b3 [id=9e38178c12314306a0b595904a402f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.412024 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb3d5e19cda48fda124b6238902c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.415673 28524 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:32.416195 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:32.416431 28524 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.417008 29445 raft_consensus.cc:2749] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:32.417222 29445 raft_consensus.cc:2749] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:32.417411 29445 consensus_queue.cc:1048] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:32.421290 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.429175 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb3d5e19cda48fda124b6238902c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.430822 28568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd1799035a3431092fbdfad3c42d22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.450902 28415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233f6a8e9369411b8e0c35a167c55ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.450971 28553 tablet_service.cc:1511] Processing CreateTablet for tablet db16d86664794305bde714a7ff4d5caf (DEFAULT_TABLE table=default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.451598 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.452036 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ae379304674bc3941af125f0d9d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.407816 29600 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.470644 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 813604f6c6c747eda18f7a01d9d8ec97 (DEFAULT_TABLE table=default.Table_65a8a44c03cd4385a65c2e752446c84c [id=81f681463e614a42b8730cace88a123a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.480177 29599 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:32.482378 29600 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.484503 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 197f1935099140a98c5eb801cf44046a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.485287 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 813604f6c6c747eda18f7a01d9d8ec97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.489976 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf4e017e2e743d7b38628d6f1e0d72e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.494006 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db16d86664794305bde714a7ff4d5caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.499059 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bbc5d5f49a04216a352ddb1b571dce0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:32.505393 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 105609us
I20260320 15:27:32.512738 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb7cbe0a9224412880144927ec8fa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.521116 29423 fs_manager.cc:714] Time spent opening directory manager: real 0.398s user 0.029s sys 0.044s
I20260320 15:27:32.529919 28586 raft_consensus.cc:3060] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:32.533586 28736 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:32.533601 29600 raft_consensus.cc:493] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.533779 28736 ts_tablet_manager.cc:1403] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.408s user 0.003s sys 0.003s
I20260320 15:27:32.533712 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:32.533954 28584 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:32.534839 29614 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:32.535048 29614 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:32.536044 29614 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
I20260320 15:27:32.536443 28736 raft_consensus.cc:359] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.536665 28736 raft_consensus.cc:385] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.536761 28736 raft_consensus.cc:740] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.536993 28736 consensus_queue.cc:260] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.537694 28736 ts_tablet_manager.cc:1434] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:32.538054 28736 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:32.537966 28427 tablet_service.cc:1511] Processing CreateTablet for tablet 4bd1799035a3431092fbdfad3c42d22b (DEFAULT_TABLE table=default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.538621 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 6 candidate_status { last_received { term: 5 index: 583 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:32.540822 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 76379bf0fa554820be020f647582a72b (DEFAULT_TABLE table=default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.542282 29600 raft_consensus.cc:515] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.543147 29600 leader_election.cc:290] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.547518 28734 tablet_bootstrap.cc:654] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.547176 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 26ae379304674bc3941af125f0d9d6b3 (DEFAULT_TABLE table=default.Table_ce843e88d7f24053b0be457f201e11ae [id=9941875dffce49d680808963e123cdbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.563228 28733 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:32.563473 28733 ts_tablet_manager.cc:1403] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.500s user 0.008s sys 0.019s
I20260320 15:27:32.564368 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (DEFAULT_TABLE table=default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.565241 28733 raft_consensus.cc:359] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.565471 28733 raft_consensus.cc:385] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.565668 28733 raft_consensus.cc:740] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.574177 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68 (DEFAULT_TABLE table=default.Table_fda3998f623540e6beca0a6f3455b0b3 [id=9e38178c12314306a0b595904a402f21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.583774 28733 consensus_queue.cc:260] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.586225 28733 ts_tablet_manager.cc:1434] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260320 15:27:32.588425 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bbc5d5f49a04216a352ddb1b571dce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.592000 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:32.596349 28302 raft_consensus.cc:2468] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:32.597659 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfbc7338eda489987315e681ee44146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.598037 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bd1799035a3431092fbdfad3c42d22b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.598280 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54c75b9deab44c829f6720480e38c80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.599049 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ae379304674bc3941af125f0d9d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.599191 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:32.601421 28300 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:32.603718 28412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4150c4c26a26405b9be16f66d15a8948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.607336 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 514e7b02a28142c2b371644089044a9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.608279 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76379bf0fa554820be020f647582a72b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:32.619554 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 95286.7us
W20260320 15:27:32.623699 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 206712us
I20260320 15:27:32.624302 28524 leader_election.cc:304] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.624389 29445 consensus_queue.cc:1048] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:32.624704 29599 raft_consensus.cc:2749] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:32.628430 29445 raft_consensus.cc:493] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.628896 29445 raft_consensus.cc:515] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.629871 29445 leader_election.cc:290] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:32.630816 28733 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:32.633437 28733 tablet_bootstrap.cc:654] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:32.640885 28109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 52144.5us
I20260320 15:27:32.642192 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:32.642592 28298 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:32.645545 28382 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:32.645931 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 67095us
I20260320 15:27:32.656986 28411 tablet_service.cc:1511] Processing CreateTablet for tablet db16d86664794305bde714a7ff4d5caf (DEFAULT_TABLE table=default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:32.657842 28524 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:32.658910 29445 raft_consensus.cc:2804] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:32.659277 29445 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:32.659456 29445 raft_consensus.cc:3060] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:32.662377 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a61cfbf6eac4665a0b12ea7ccb9bb68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.663602 29600 raft_consensus.cc:2804] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:32.663566 28408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20efd94b09746c88bb48ba36af4759e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.663870 29608 consensus_queue.cc:1048] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.069s
I20260320 15:27:32.663775 29600 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260320 15:27:32.664458 28109 leader_election.cc:336] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111)
I20260320 15:27:32.664714 28109 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:32.665118 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594357400f8c44138c7c4f11bf6d5afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.665148 29614 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260320 15:27:32.669762 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:32.671201 28736 tablet_bootstrap.cc:654] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:32.673975 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:32.678822 29608 consensus_queue.cc:1048] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260320 15:27:32.679332 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4190329df1b644c392ff8cbace9aae7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.680627 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:32.680953 29600 raft_consensus.cc:3060] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:32.685531 29553 raft_consensus.cc:493] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.685808 29553 raft_consensus.cc:515] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.686761 29553 leader_election.cc:290] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.687422 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3422cf33324ab89a2b44e27cae0eb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.688961 28524 leader_election.cc:304] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.694069 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db16d86664794305bde714a7ff4d5caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.694111 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:32.698025 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:32.699267 28240 leader_election.cc:343] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.700075 29445 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.703095 28410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48c1002cbdd845e2b96dcb57c450cc6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:32.706252 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:32.711437 29445 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:32.713475 28239 leader_election.cc:343] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.713698 28239 leader_election.cc:304] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.714140 29445 raft_consensus.cc:2749] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:32.718267 28873 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
I20260320 15:27:32.719990 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:32.723730 29629 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260320 15:27:32.725041 29553 raft_consensus.cc:2749] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:32.726573 28382 leader_election.cc:343] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.729641 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:32.729874 28298 raft_consensus.cc:3060] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:32.735074 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:32.751361 29423 fs_manager.cc:730] Time spent opening block manager: real 0.152s user 0.003s sys 0.015s
I20260320 15:27:32.759337 28736 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:32.759606 28736 ts_tablet_manager.cc:1403] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.222s user 0.002s sys 0.014s
I20260320 15:27:32.760509 28736 raft_consensus.cc:359] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.760743 28736 raft_consensus.cc:385] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.760838 28736 raft_consensus.cc:740] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.761111 28736 consensus_queue.cc:260] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.761772 28736 ts_tablet_manager.cc:1434] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:32.762084 28733 tablet_bootstrap.cc:492] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:32.762277 28733 ts_tablet_manager.cc:1403] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.150s user 0.007s sys 0.007s
I20260320 15:27:32.758798 29423 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data,/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
uuid: "07e8bb8819eb4cb4892e0cba1f758892"
format_stamp: "Formatted at 2026-03-20 15:27:14 on dist-test-slave-qjxl"
I20260320 15:27:32.765072 28733 raft_consensus.cc:359] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.765520 28733 raft_consensus.cc:385] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.765614 28733 raft_consensus.cc:740] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.765908 28733 consensus_queue.cc:260] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.767688 28736 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:32.769678 28736 tablet_bootstrap.cc:654] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.771018 28733 ts_tablet_manager.cc:1434] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260320 15:27:32.773298 28733 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:32.775415 28733 tablet_bootstrap.cc:654] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.790207 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 6292904782fd4b619483135e58b66684
I20260320 15:27:32.790881 28904 catalog_manager.cc:6741] ReplaceTablet: tablet 6292904782fd4b619483135e58b66684 deleted and replaced by tablet 92ea5e924a734c8ba6dfb641359593a3
I20260320 15:27:32.791054 28904 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 48) took 2042 ms. Trace:
I20260320 15:27:32.791263 28557 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:32.791386 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:32.791163 28904 rpcz_store.cc:276] 0320 15:27:30.748346 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.156551 (+408205us) service_pool.cc:225] Handling call
0320 15:27:31.178082 (+ 21531us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:32.790993 (+1612911us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:32.732195 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:32.732557 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:32.732565 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:32.732567 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:32.735354 (+ 2787us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:32.735378 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:32.735387 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:32.735515 (+ 128us) tablet.cc:700] Row locks acquired
0320 15:27:32.735518 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:32.735601 (+ 83us) write_op.cc:270] Start()
0320 15:27:32.735655 (+ 54us) write_op.cc:276] Timestamp: P: 1774020452735578 usec, L: 0
0320 15:27:32.735661 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:32.766276 (+ 30615us) log.cc:844] Serialized 622 byte log entry
0320 15:27:32.767050 (+ 774us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:32.789091 (+ 22041us) op_driver.cc:464] REPLICATION: finished
0320 15:27:32.789227 (+ 136us) write_op.cc:301] APPLY: starting
0320 15:27:32.789250 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:32.789314 (+ 64us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:32.789319 (+ 5us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:32.789510 (+ 191us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:32.789591 (+ 81us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:32.789602 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:32.789712 (+ 110us) log.cc:844] Serialized 29 byte log entry
0320 15:27:32.789797 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.789830 (+ 33us) write_op.cc:454] Released schema lock
0320 15:27:32.789840 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":48489216,"child_traces":[["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":715,"apply.run_wall_time_us":713,"num_ops":2,"prepare.queue_time_us":110962,"prepare.run_cpu_time_us":2122,"prepare.run_wall_time_us":34964,"raft.queue_time_us":12814,"raft.run_cpu_time_us":557,"raft.run_wall_time_us":12560,"replication_time_us":53380,"spinlock_wait_cycles":70214656,"thread_start_us":110503,"threads_started":1}]]}
I20260320 15:27:32.764952 29423 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260320 15:27:32.799927 28298 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:32.802425 28524 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.802939 29445 raft_consensus.cc:2804] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:32.803100 29445 raft_consensus.cc:697] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:32.803453 29445 consensus_queue.cc:237] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:32.802824 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 164112us
I20260320 15:27:32.811383 28734 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:32.804566 29600 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.810415 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:32.814659 28734 ts_tablet_manager.cc:1403] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.416s user 0.006s sys 0.010s
I20260320 15:27:32.814965 29600 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.818838 28446 raft_consensus.cc:2393] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:32.821732 28383 leader_election.cc:304] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:32.824390 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:32.824643 29245 raft_consensus.cc:2804] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:32.824769 28298 raft_consensus.cc:2393] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:32.824767 29245 raft_consensus.cc:493] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:32.826982 29245 raft_consensus.cc:3060] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:32.844370 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:32.844770 28734 raft_consensus.cc:359] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.846839 28734 raft_consensus.cc:385] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.847015 28734 raft_consensus.cc:740] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.849220 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:32.850728 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:32.848313 28734 consensus_queue.cc:260] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.853458 28382 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:32.855466 29626 raft_consensus.cc:2749] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:32.858619 28734 ts_tablet_manager.cc:1434] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.038s user 0.002s sys 0.001s
I20260320 15:27:32.858850 28733 tablet_bootstrap.cc:492] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:32.858980 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 6292904782fd4b619483135e58b66684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6292904782fd4b619483135e58b66684 already in progress: creating tablet
I20260320 15:27:32.859041 28733 ts_tablet_manager.cc:1403] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.086s user 0.003s sys 0.003s
I20260320 15:27:32.856594 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:32.859942 28733 raft_consensus.cc:359] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.860149 28733 raft_consensus.cc:385] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.860266 28733 raft_consensus.cc:740] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.860535 28733 consensus_queue.cc:260] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.860649 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:32.862188 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:32.862566 28733 ts_tablet_manager.cc:1434] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:32.864037 28905 catalog_manager.cc:5671] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:32.873121 28418 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 37) took 1020 ms. Trace:
I20260320 15:27:32.874327 28734 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:32.876679 28414 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 35) took 1040 ms. Trace:
I20260320 15:27:32.876920 28414 rpcz_store.cc:276] 0320 15:27:31.825369 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.826615 (+ 1246us) service_pool.cc:225] Handling call
0320 15:27:31.960079 (+133464us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.960118 (+ 39us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.825400 (+865282us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.865557 (+ 40157us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.865747 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.877130 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:32.877393 28905 catalog_manager.cc:5671] T 48fa326348104d148abbba907f4c547f P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:32.877956 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:32.878259 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:32.878798 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:32.879397 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
W20260320 15:27:32.880843 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6292904782fd4b619483135e58b66684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6292904782fd4b619483135e58b66684 already in progress: creating tablet
I20260320 15:27:32.881449 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_9234513d2cd54f55bf709a1c6fb0de1c [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 1 got 0
I20260320 15:27:32.881516 28425 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 29) took 1095 ms. Trace:
I20260320 15:27:32.881666 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:32.881615 28425 rpcz_store.cc:276] 0320 15:27:31.777658 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.777871 (+ 213us) service_pool.cc:225] Handling call
0320 15:27:31.837441 (+ 59570us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.837473 (+ 32us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.682667 (+845194us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.872830 (+190163us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.873006 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.881799 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:32.882004 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:32.882253 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]). Expected version 1 got 0
I20260320 15:27:32.882442 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:32.882608 28408 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 44) took 1036 ms. Trace:
I20260320 15:27:32.882717 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:32.882841 28733 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:32.882903 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:32.882756 28408 rpcz_store.cc:276] 0320 15:27:31.833734 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.833841 (+ 107us) service_pool.cc:225] Handling call
0320 15:27:31.854058 (+ 20217us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.854098 (+ 40us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.835319 (+981221us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.870117 (+ 34798us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.870302 (+ 185us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.883225 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:32.883276 29445 raft_consensus.cc:493] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.883584 29445 raft_consensus.cc:515] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.884562 28418 rpcz_store.cc:276] 0320 15:27:31.825612 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.826672 (+ 1060us) service_pool.cc:225] Handling call
0320 15:27:31.862651 (+ 35979us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.862682 (+ 31us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.751892 (+889210us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.839461 (+ 87569us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.846233 (+ 6772us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.884831 29618 raft_consensus.cc:493] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.885023 29618 raft_consensus.cc:515] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.885269 29445 leader_election.cc:290] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:32.886085 29618 leader_election.cc:290] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.888142 28419 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 28) took 1099 ms. Trace:
W20260320 15:27:32.891366 28524 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 66a9ddea49e641da881fba339ddef549
W20260320 15:27:32.891674 28524 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 66a9ddea49e641da881fba339ddef549
I20260320 15:27:32.891805 28524 leader_election.cc:304] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.892298 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:32.892946 28239 leader_election.cc:343] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.895601 29245 raft_consensus.cc:515] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.896970 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:32.897162 29245 leader_election.cc:290] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:32.897841 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:32.898164 28298 raft_consensus.cc:3060] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:32.898911 28382 leader_election.cc:343] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.898839 28419 rpcz_store.cc:276] 0320 15:27:31.777517 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.789562 (+ 12045us) service_pool.cc:225] Handling call
0320 15:27:31.837380 (+ 47818us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.837413 (+ 33us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.746162 (+908749us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.876330 (+130168us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.876539 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.883476 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:32.900094 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:32.900576 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:32.900910 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:32.901186 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:32.901460 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:32.901706 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 1 got 0
I20260320 15:27:32.901757 28733 tablet_bootstrap.cc:654] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.902184 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:32.902630 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:32.902627 28905 catalog_manager.cc:5671] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:32.903234 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
W20260320 15:27:32.904980 28240 leader_election.cc:343] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:32.905177 28240 leader_election.cc:304] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:32.905592 29618 raft_consensus.cc:2749] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:32.906335 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6292904782fd4b619483135e58b66684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6292904782fd4b619483135e58b66684 already in progress: creating tablet
I20260320 15:27:32.909611 28426 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 31) took 1075 ms. Trace:
I20260320 15:27:32.914525 28426 rpcz_store.cc:276] 0320 15:27:31.823119 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.824667 (+ 1548us) service_pool.cc:225] Handling call
0320 15:27:32.252996 (+428329us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:32.253032 (+ 36us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.842027 (+588995us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.894454 (+ 52427us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.898111 (+ 3657us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.916709 28413 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 39) took 1088 ms. Trace:
I20260320 15:27:32.916843 28413 rpcz_store.cc:276] 0320 15:27:31.825806 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.826729 (+ 923us) service_pool.cc:225] Handling call
0320 15:27:31.845221 (+ 18492us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.845242 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.806847 (+961605us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.908569 (+101722us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.914232 (+ 5663us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.919044 28734 tablet_bootstrap.cc:654] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.919390 28733 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:32.919598 28733 ts_tablet_manager.cc:1403] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.001s
I20260320 15:27:32.920473 28733 raft_consensus.cc:359] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.920709 28733 raft_consensus.cc:385] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.920785 28733 raft_consensus.cc:740] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.921053 28733 consensus_queue.cc:260] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.921183 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 07ff84dedc074dabb4e8e2d8460066e7
I20260320 15:27:32.921663 28733 ts_tablet_manager.cc:1434] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:32.921803 28902 catalog_manager.cc:6741] ReplaceTablet: tablet 07ff84dedc074dabb4e8e2d8460066e7 deleted and replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a
I20260320 15:27:32.921949 28902 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 49) took 2151 ms. Trace:
I20260320 15:27:32.922040 28733 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:32.922039 28902 rpcz_store.cc:276] 0320 15:27:30.770342 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.181335 (+410993us) service_pool.cc:225] Handling call
0320 15:27:31.200021 (+ 18686us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:32.921888 (+1721867us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:32.914262 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:32.914393 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:32.914399 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:32.914401 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:32.914520 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:32.914535 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:32.914540 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:32.914639 (+ 99us) tablet.cc:700] Row locks acquired
0320 15:27:32.914641 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:32.914673 (+ 32us) write_op.cc:270] Start()
0320 15:27:32.914706 (+ 33us) write_op.cc:276] Timestamp: P: 1774020452914645 usec, L: 0
0320 15:27:32.914710 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:32.914899 (+ 189us) log.cc:844] Serialized 622 byte log entry
0320 15:27:32.919040 (+ 4141us) op_driver.cc:464] REPLICATION: finished
0320 15:27:32.919337 (+ 297us) write_op.cc:301] APPLY: starting
0320 15:27:32.919356 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:32.919411 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:32.919450 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:32.919701 (+ 251us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:32.919775 (+ 74us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:32.919783 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:32.919882 (+ 99us) log.cc:844] Serialized 29 byte log entry
0320 15:27:32.919950 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.919973 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:32.919980 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:32.920054 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.920058 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":32911744,"child_traces":[["op",{"apply.queue_time_us":229,"apply.run_cpu_time_us":632,"apply.run_wall_time_us":713,"num_ops":2,"prepare.queue_time_us":156,"prepare.run_cpu_time_us":744,"prepare.run_wall_time_us":744,"raft.queue_time_us":487,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":287,"replication_time_us":4293}]]}
I20260320 15:27:32.923969 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:32.924770 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 07ff84dedc074dabb4e8e2d8460066e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07ff84dedc074dabb4e8e2d8460066e7 already in progress: creating tablet
I20260320 15:27:32.927687 28733 tablet_bootstrap.cc:654] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.930696 28557 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:32.936851 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 07ff84dedc074dabb4e8e2d8460066e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07ff84dedc074dabb4e8e2d8460066e7 already in progress: creating tablet
I20260320 15:27:32.892946 29445 raft_consensus.cc:2749] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:32.941737 28736 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:32.941943 28736 ts_tablet_manager.cc:1403] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.174s user 0.006s sys 0.010s
I20260320 15:27:32.943104 28736 raft_consensus.cc:359] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.943397 28736 raft_consensus.cc:385] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.943732 28736 raft_consensus.cc:740] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.944272 28736 consensus_queue.cc:260] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.945158 28736 ts_tablet_manager.cc:1434] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:32.945771 28736 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:32.949842 28736 tablet_bootstrap.cc:654] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:32.951758 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:32.952749 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 07ff84dedc074dabb4e8e2d8460066e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07ff84dedc074dabb4e8e2d8460066e7 already in progress: creating tablet
I20260320 15:27:32.954881 28410 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:55030 (request call id 42) took 1128 ms. Trace:
I20260320 15:27:32.955090 28410 rpcz_store.cc:276] 0320 15:27:31.826111 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:31.826992 (+ 881us) service_pool.cc:225] Handling call
0320 15:27:31.861271 (+ 34279us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0320 15:27:31.861311 (+ 40us) ts_tablet_manager.cc:682] Creating new metadata...
0320 15:27:32.889523 (+1028212us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:32.954619 (+ 65096us) tablet_replica.cc:206] Creating consensus instance
0320 15:27:32.954786 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:32.957778 28298 raft_consensus.cc:2468] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:32.958714 28383 leader_election.cc:304] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:32.959195 29245 raft_consensus.cc:2804] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:32.960005 29245 raft_consensus.cc:697] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:32.960640 29245 consensus_queue.cc:237] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.978255 28734 tablet_bootstrap.cc:492] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:32.978492 28734 ts_tablet_manager.cc:1403] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.119s user 0.007s sys 0.002s
I20260320 15:27:32.979727 28734 raft_consensus.cc:359] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.979960 28734 raft_consensus.cc:385] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.980062 28734 raft_consensus.cc:740] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.980361 28734 consensus_queue.cc:260] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.980999 28734 ts_tablet_manager.cc:1434] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:32.983160 28736 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:32.983613 28736 ts_tablet_manager.cc:1403] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.002s
I20260320 15:27:32.985721 28734 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:32.985394 28736 raft_consensus.cc:359] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.986531 28736 raft_consensus.cc:385] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:32.986735 28736 raft_consensus.cc:740] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:32.987182 28736 consensus_queue.cc:260] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:32.987396 29445 raft_consensus.cc:493] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:32.993712 28890 catalog_manager.cc:5671] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:32.995075 28736 ts_tablet_manager.cc:1434] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.000s sys 0.003s
I20260320 15:27:32.996174 28736 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:32.999115 28736 tablet_bootstrap.cc:654] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.001169 28734 tablet_bootstrap.cc:654] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.001593 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:33.003970 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:33.004626 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:33.005028 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:33.005710 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:33.005990 29618 raft_consensus.cc:493] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.006219 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:33.006199 29618 raft_consensus.cc:515] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.007189 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:33.007441 29618 leader_election.cc:290] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.008450 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
W20260320 15:27:33.008922 28240 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:33.009421 29445 raft_consensus.cc:515] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:33.011041 28239 leader_election.cc:343] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
I20260320 15:27:33.011298 28239 leader_election.cc:304] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:33.011698 29618 raft_consensus.cc:2749] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:33.011961 29445 leader_election.cc:290] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.013716 28734 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:33.014489 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
W20260320 15:27:33.015756 28524 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b8f3ee3c611d41b483bd35f868042d05
W20260320 15:27:33.016134 28524 leader_election.cc:343] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:33.016291 28736 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:33.015373 28734 ts_tablet_manager.cc:1403] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.001s
I20260320 15:27:33.016449 28736 ts_tablet_manager.cc:1403] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.001s
I20260320 15:27:33.016438 28524 leader_election.cc:304] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:33.017553 28736 raft_consensus.cc:359] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.017792 28736 raft_consensus.cc:385] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.017758 28734 raft_consensus.cc:359] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.017899 28736 raft_consensus.cc:740] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.018013 28734 raft_consensus.cc:385] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.018110 28734 raft_consensus.cc:740] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.018162 28736 consensus_queue.cc:260] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.018411 28733 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:33.018617 28733 ts_tablet_manager.cc:1403] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.097s user 0.021s sys 0.028s
I20260320 15:27:33.018776 28736 ts_tablet_manager.cc:1434] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:33.019277 28736 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:33.019790 28733 raft_consensus.cc:359] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.020299 28733 raft_consensus.cc:385] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.020666 28733 raft_consensus.cc:740] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.021046 28733 consensus_queue.cc:260] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.021664 29617 raft_consensus.cc:2749] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:33.022010 28736 tablet_bootstrap.cc:654] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.022075 28733 ts_tablet_manager.cc:1434] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:33.022708 28733 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:33.026480 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:33.026960 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:33.027163 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]). Expected version 1 got 0
I20260320 15:27:33.027360 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:33.027876 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:33.028656 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:33.029424 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:33.029856 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:33.030148 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:33.030660 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:33.031073 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:33.031390 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:33.031657 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:33.031872 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:33.032311 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:33.032603 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 1 got 0
I20260320 15:27:33.033087 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:33.033372 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:33.030257 28734 consensus_queue.cc:260] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.036032 28733 tablet_bootstrap.cc:654] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.053565 28734 ts_tablet_manager.cc:1434] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.037s user 0.002s sys 0.002s
I20260320 15:27:33.056564 29599 raft_consensus.cc:493] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.056872 29599 raft_consensus.cc:515] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.058027 29599 leader_election.cc:290] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.058528 28733 tablet_bootstrap.cc:492] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:33.058612 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0242a7968541f480367b45cda5d9cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.058753 28733 ts_tablet_manager.cc:1403] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.000s
I20260320 15:27:33.059827 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0242a7968541f480367b45cda5d9cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.060070 28298 raft_consensus.cc:2468] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:33.060658 28733 raft_consensus.cc:359] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.060889 28524 leader_election.cc:304] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.061019 28733 raft_consensus.cc:385] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.061110 28733 raft_consensus.cc:740] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.061234 29599 raft_consensus.cc:2804] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.061401 29599 raft_consensus.cc:493] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260320 15:27:33.061661 28524 leader_election.cc:343] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.061729 29599 raft_consensus.cc:3060] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.063305 28734 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:33.064911 28733 consensus_queue.cc:260] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.080422 28734 tablet_bootstrap.cc:654] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.089646 28733 ts_tablet_manager.cc:1434] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.031s user 0.003s sys 0.001s
I20260320 15:27:33.090390 28733 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:33.093096 28733 tablet_bootstrap.cc:654] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.109679 28734 tablet_bootstrap.cc:492] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:33.110047 28734 ts_tablet_manager.cc:1403] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.008s
I20260320 15:27:33.111107 28734 raft_consensus.cc:359] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.111263 29617 raft_consensus.cc:493] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.111460 28734 raft_consensus.cc:385] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.111608 28734 raft_consensus.cc:740] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.111474 29617 raft_consensus.cc:515] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.111920 28734 consensus_queue.cc:260] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.112516 29617 leader_election.cc:290] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.113821 28736 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:33.114104 28736 ts_tablet_manager.cc:1403] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.095s user 0.033s sys 0.049s
I20260320 15:27:33.114344 28733 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:33.114521 28733 ts_tablet_manager.cc:1403] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.004s
I20260320 15:27:33.114777 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.115224 28736 raft_consensus.cc:359] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.115429 28736 raft_consensus.cc:385] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.115504 28736 raft_consensus.cc:740] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.115854 28733 raft_consensus.cc:359] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.116066 28733 raft_consensus.cc:385] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.116021 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a18f32c29224bd883ceb20658dff228" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.116168 28733 raft_consensus.cc:740] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.116407 28733 consensus_queue.cc:260] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.117141 28733 ts_tablet_manager.cc:1434] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:33.115782 28736 consensus_queue.cc:260] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.119923 28734 ts_tablet_manager.cc:1434] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:33.121431 28410 tablet_service.cc:1511] Processing CreateTablet for tablet afb692aab24849bc8e50086927421e13 (DEFAULT_TABLE table=default.Table_ea1f04fcc8eb448cac5344c72e695c99 [id=3d899b93c9044d4fa6501ac819a26533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.121995 28410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb692aab24849bc8e50086927421e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.128762 28265 tablet_service.cc:1511] Processing CreateTablet for tablet aba79ddde6e842dfb67597c5d9e371cb (DEFAULT_TABLE table=default.Table_78b650f7f9354416913f3c232c22eb59 [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.128901 28555 tablet_service.cc:1511] Processing CreateTablet for tablet aba79ddde6e842dfb67597c5d9e371cb (DEFAULT_TABLE table=default.Table_78b650f7f9354416913f3c232c22eb59 [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.129370 28265 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba79ddde6e842dfb67597c5d9e371cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.129406 28284 tablet_service.cc:1511] Processing CreateTablet for tablet 50c1398534d84701ae7085a037f236ad (DEFAULT_TABLE table=default.Table_c081374356e242d6bda444f6aac87050 [id=71ce933a16db42d6a03ddd1fdcd93fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.129318 28281 tablet_service.cc:1511] Processing CreateTablet for tablet a6498952e19c45dbacb1fa7af8ece8f8 (DEFAULT_TABLE table=default.Table_e2c5841a2d914105ac376afba3c85cb1 [id=82139981fe9e40709b94e58df3f4845b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.129799 28284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c1398534d84701ae7085a037f236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.130059 28268 tablet_service.cc:1511] Processing CreateTablet for tablet afb692aab24849bc8e50086927421e13 (DEFAULT_TABLE table=default.Table_ea1f04fcc8eb448cac5344c72e695c99 [id=3d899b93c9044d4fa6501ac819a26533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.131317 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba3def9ab7641d2ad4a6d0538bf978a (DEFAULT_TABLE table=default.Table_30673191803644ec9a6dba7a6ec38dde [id=df0b3e622bbf41a2b487d7f4c969f022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.130112 28270 tablet_service.cc:1511] Processing CreateTablet for tablet fca60a0f372f4d1194d754d124b78ff8 (DEFAULT_TABLE table=default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.131922 28552 tablet_service.cc:1511] Processing CreateTablet for tablet a6498952e19c45dbacb1fa7af8ece8f8 (DEFAULT_TABLE table=default.Table_e2c5841a2d914105ac376afba3c85cb1 [id=82139981fe9e40709b94e58df3f4845b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.132582 28565 tablet_service.cc:1511] Processing CreateTablet for tablet fca60a0f372f4d1194d754d124b78ff8 (DEFAULT_TABLE table=default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.133035 28567 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba3def9ab7641d2ad4a6d0538bf978a (DEFAULT_TABLE table=default.Table_30673191803644ec9a6dba7a6ec38dde [id=df0b3e622bbf41a2b487d7f4c969f022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.133271 28524 leader_election.cc:304] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:33.136022 28419 tablet_service.cc:1511] Processing CreateTablet for tablet fca60a0f372f4d1194d754d124b78ff8 (DEFAULT_TABLE table=default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.136615 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb692aab24849bc8e50086927421e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.138463 28419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca60a0f372f4d1194d754d124b78ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.139621 28414 tablet_service.cc:1511] Processing CreateTablet for tablet a185f84c24ea402684aa1cc3628d4f11 (DEFAULT_TABLE table=default.Table_8b5b70bee34f4caabd4a276b20f57793 [id=e4f2c5eb7fff4a718a8cdbbb8b72d953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.140100 28414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a185f84c24ea402684aa1cc3628d4f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.141893 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6498952e19c45dbacb1fa7af8ece8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.141929 28423 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba3def9ab7641d2ad4a6d0538bf978a (DEFAULT_TABLE table=default.Table_30673191803644ec9a6dba7a6ec38dde [id=df0b3e622bbf41a2b487d7f4c969f022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.142436 28420 tablet_service.cc:1511] Processing CreateTablet for tablet aba79ddde6e842dfb67597c5d9e371cb (DEFAULT_TABLE table=default.Table_78b650f7f9354416913f3c232c22eb59 [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.143043 28736 ts_tablet_manager.cc:1434] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.029s user 0.002s sys 0.001s
I20260320 15:27:33.143040 28270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca60a0f372f4d1194d754d124b78ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.142805 28734 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:33.145344 28736 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:33.149693 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.151422 28427 tablet_service.cc:1511] Processing CreateTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 (DEFAULT_TABLE table=default.Table_9b2e32517f9d49a9a117be78ffddb91e [id=5df94a1a6fc04a4983dc2888ef98c2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.144636 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 4d12bd6ae67f4600a0d54e7d5d502dda (DEFAULT_TABLE table=default.Table_108f31bd087a4a56bd7916340d021c75 [id=ebfa27643f0d479fb92e6a7b2cadd878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.152408 28408 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2e17c67c544e38840c5dd8dec32e5 (DEFAULT_TABLE table=default.Table_38d617af0e3e4149b252aac03d1863ab [id=d9aba8f21a9f469ebd3710a32d188f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.152658 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.146482 28272 tablet_service.cc:1511] Processing CreateTablet for tablet e779a274913f4caeaf82e3f180457830 (DEFAULT_TABLE table=default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.154239 28277 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2e17c67c544e38840c5dd8dec32e5 (DEFAULT_TABLE table=default.Table_38d617af0e3e4149b252aac03d1863ab [id=d9aba8f21a9f469ebd3710a32d188f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.155299 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 50c1398534d84701ae7085a037f236ad (DEFAULT_TABLE table=default.Table_c081374356e242d6bda444f6aac87050 [id=71ce933a16db42d6a03ddd1fdcd93fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.155512 29626 raft_consensus.cc:493] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.151902 28426 tablet_service.cc:1511] Processing CreateTablet for tablet a6498952e19c45dbacb1fa7af8ece8f8 (DEFAULT_TABLE table=default.Table_e2c5841a2d914105ac376afba3c85cb1 [id=82139981fe9e40709b94e58df3f4845b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.155375 29245 raft_consensus.cc:493] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.156152 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:33.156374 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:33.156597 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:33.156809 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba3def9ab7641d2ad4a6d0538bf978a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.157224 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:33.157617 28283 tablet_service.cc:1511] Processing CreateTablet for tablet a185f84c24ea402684aa1cc3628d4f11 (DEFAULT_TABLE table=default.Table_8b5b70bee34f4caabd4a276b20f57793 [id=e4f2c5eb7fff4a718a8cdbbb8b72d953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.158375 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:33.160472 29617 raft_consensus.cc:2749] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:33.160398 28553 tablet_service.cc:1511] Processing CreateTablet for tablet 50c1398534d84701ae7085a037f236ad (DEFAULT_TABLE table=default.Table_c081374356e242d6bda444f6aac87050 [id=71ce933a16db42d6a03ddd1fdcd93fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.163630 29599 raft_consensus.cc:515] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.164363 28551 tablet_service.cc:1511] Processing CreateTablet for tablet a185f84c24ea402684aa1cc3628d4f11 (DEFAULT_TABLE table=default.Table_8b5b70bee34f4caabd4a276b20f57793 [id=e4f2c5eb7fff4a718a8cdbbb8b72d953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.164439 28557 tablet_service.cc:1511] Processing CreateTablet for tablet afb692aab24849bc8e50086927421e13 (DEFAULT_TABLE table=default.Table_ea1f04fcc8eb448cac5344c72e695c99 [id=3d899b93c9044d4fa6501ac819a26533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.164418 28566 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2e17c67c544e38840c5dd8dec32e5 (DEFAULT_TABLE table=default.Table_38d617af0e3e4149b252aac03d1863ab [id=d9aba8f21a9f469ebd3710a32d188f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.165441 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e779a274913f4caeaf82e3f180457830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.165408 28266 tablet_service.cc:1511] Processing CreateTablet for tablet ef0d80754dfd4bbd9817f568063511ef (DEFAULT_TABLE table=default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.165882 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:33.166097 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.166812 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:33.170742 28902 catalog_manager.cc:5671] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } }
I20260320 15:27:33.171825 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:33.173198 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:33.174579 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:33.175462 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:33.176546 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:33.177850 28271 tablet_service.cc:1511] Processing CreateTablet for tablet d04ab44039124f36ac60601098586b4e (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.177904 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.179174 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:33.179986 29423 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260320 15:27:33.180076 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:33.181135 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]). Expected version 1 got 0
I20260320 15:27:33.181749 29423 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260320 15:27:33.182225 29423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260320 15:27:33.182102 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:33.182612 28733 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:33.182544 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.182680 28282 tablet_service.cc:1511] Processing CreateTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 (DEFAULT_TABLE table=default.Table_9b2e32517f9d49a9a117be78ffddb91e [id=5df94a1a6fc04a4983dc2888ef98c2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.185087 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 4d12bd6ae67f4600a0d54e7d5d502dda (DEFAULT_TABLE table=default.Table_108f31bd087a4a56bd7916340d021c75 [id=ebfa27643f0d479fb92e6a7b2cadd878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.185483 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0242a7968541f480367b45cda5d9cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.183990 28560 tablet_service.cc:1511] Processing CreateTablet for tablet ef0d80754dfd4bbd9817f568063511ef (DEFAULT_TABLE table=default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.186784 28298 raft_consensus.cc:3060] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.187873 28559 tablet_service.cc:1511] Processing CreateTablet for tablet d04ab44039124f36ac60601098586b4e (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.188576 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 (DEFAULT_TABLE table=default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.188660 28563 tablet_service.cc:1511] Processing CreateTablet for tablet 4d12bd6ae67f4600a0d54e7d5d502dda (DEFAULT_TABLE table=default.Table_108f31bd087a4a56bd7916340d021c75 [id=ebfa27643f0d479fb92e6a7b2cadd878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.193727 28558 tablet_service.cc:1511] Processing CreateTablet for tablet e779a274913f4caeaf82e3f180457830 (DEFAULT_TABLE table=default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.195281 29445 raft_consensus.cc:493] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.198800 29599 leader_election.cc:290] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.198843 28556 tablet_service.cc:1511] Processing CreateTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 (DEFAULT_TABLE table=default.Table_9b2e32517f9d49a9a117be78ffddb91e [id=5df94a1a6fc04a4983dc2888ef98c2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.198800 29617 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.198818 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba79ddde6e842dfb67597c5d9e371cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.199800 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0d80754dfd4bbd9817f568063511ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.200554 28282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16ffb28029e4b7ca37ce36efe9e0271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.201349 29617 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.184585 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:33.205205 28565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca60a0f372f4d1194d754d124b78ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.207247 28567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba3def9ab7641d2ad4a6d0538bf978a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.207832 28553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c1398534d84701ae7085a037f236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.208259 29617 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.209053 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ab44039124f36ac60601098586b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.206996 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:33.210464 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:33.214476 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:33.215566 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:33.216666 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:33.217469 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d12bd6ae67f4600a0d54e7d5d502dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.218158 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:33.219393 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:33.220999 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:33.222438 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6498952e19c45dbacb1fa7af8ece8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.222560 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:33.208400 29445 raft_consensus.cc:515] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.224295 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:33.225106 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:33.225651 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 1 got 0
I20260320 15:27:33.226250 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:33.227447 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:33.228116 28566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2e17c67c544e38840c5dd8dec32e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.228919 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:33.229094 28733 tablet_bootstrap.cc:654] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.229537 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:33.229808 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0d80754dfd4bbd9817f568063511ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.231331 29445 leader_election.cc:290] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.232723 28558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e779a274913f4caeaf82e3f180457830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.239616 28563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d12bd6ae67f4600a0d54e7d5d502dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.242245 29445 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.246824 29445 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.250896 28557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb692aab24849bc8e50086927421e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.253979 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f94140a5e8b4a06890bca86456d64a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.255640 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16ffb28029e4b7ca37ce36efe9e0271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.255446 29445 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.264066 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a185f84c24ea402684aa1cc3628d4f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.274111 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ab44039124f36ac60601098586b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.276373 29553 raft_consensus.cc:493] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.279690 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.281387 28300 raft_consensus.cc:2468] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:33.279481 29553 raft_consensus.cc:515] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.283659 28524 leader_election.cc:304] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.284173 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.284133 28283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a185f84c24ea402684aa1cc3628d4f11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.285993 29445 raft_consensus.cc:2804] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.286199 29445 raft_consensus.cc:493] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.288354 28301 raft_consensus.cc:2468] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:33.289557 29553 leader_election.cc:290] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.290274 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2e17c67c544e38840c5dd8dec32e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.290537 28524 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.294976 29445 raft_consensus.cc:3060] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.295861 28736 tablet_bootstrap.cc:654] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.296586 29617 raft_consensus.cc:2804] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.296729 29617 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.297505 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.297922 29617 raft_consensus.cc:3060] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.299978 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:33.300262 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:33.300644 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:33.300997 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:33.301678 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:33.302686 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:33.303601 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:33.304026 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:33.305444 28585 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f1a65661949258f521dc847e674cd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:33.310807 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f94140a5e8b4a06890bca86456d64a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.311976 28422 tablet_service.cc:1511] Processing CreateTablet for tablet ef0d80754dfd4bbd9817f568063511ef (DEFAULT_TABLE table=default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.312386 29626 raft_consensus.cc:515] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.315783 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:33.316378 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:33.316658 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:33.316979 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:33.317876 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:33.318176 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:33.318425 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:33.318804 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:33.319013 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:33.319468 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]). Expected version 1 got 0
I20260320 15:27:33.319748 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:33.320022 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:33.320328 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:33.320525 28585 raft_consensus.cc:2468] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:33.320598 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:33.320935 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:33.321321 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:33.321556 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:33.321779 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:33.321931 28239 leader_election.cc:304] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.322142 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:33.322347 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_29cefe9b80e84240960c1e41f7401aaa [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 1 got 0
I20260320 15:27:33.322835 28904 catalog_manager.cc:5671] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:33.323709 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
W20260320 15:27:33.324460 28901 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7bb680bb0863485aaad85a749221e81b consensus state has no leader
I20260320 15:27:33.324709 28901 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 30) took 3320 ms. Trace:
I20260320 15:27:33.324831 28901 rpcz_store.cc:276] 0320 15:27:30.004459 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.555825 (+551366us) service_pool.cc:225] Handling call
0320 15:27:30.618850 (+ 63025us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:30.619077 (+ 227us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:30.619307 (+ 230us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:30.623702 (+ 4395us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:30.623808 (+ 106us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:30.698523 (+ 74715us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:33.324641 (+2626118us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:30.629343 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:30.629462 (+ 119us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:30.629468 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:30.629471 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:30.629577 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:30.629589 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:30.629594 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:30.629652 (+ 58us) tablet.cc:700] Row locks acquired
0320 15:27:30.629655 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:30.629692 (+ 37us) write_op.cc:270] Start()
0320 15:27:30.629726 (+ 34us) write_op.cc:276] Timestamp: P: 1774020450629629 usec, L: 0
0320 15:27:30.629730 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:30.629906 (+ 176us) log.cc:844] Serialized 405 byte log entry
0320 15:27:30.695735 (+ 65829us) op_driver.cc:464] REPLICATION: finished
0320 15:27:30.696112 (+ 377us) write_op.cc:301] APPLY: starting
0320 15:27:30.696126 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:30.696175 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:30.696198 (+ 23us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:30.696235 (+ 37us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:30.696263 (+ 28us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:30.696269 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:30.696318 (+ 49us) log.cc:844] Serialized 23 byte log entry
0320 15:27:30.696368 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:30.696390 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:30.696398 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:30.696475 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:30.696479 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":144391424,"child_traces":[["op",{"apply.queue_time_us":328,"apply.run_cpu_time_us":352,"apply.run_wall_time_us":353,"num_ops":1,"prepare.queue_time_us":4860,"prepare.run_cpu_time_us":673,"prepare.run_wall_time_us":1040,"raft.queue_time_us":37,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":65981}]]}
I20260320 15:27:33.342447 29445 raft_consensus.cc:515] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.343667 29445 leader_election.cc:290] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.347332 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.351857 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 54266dd376164673bffe829dd515b83d
I20260320 15:27:33.352706 28889 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 45) took 2642 ms. Trace:
I20260320 15:27:33.353425 28301 raft_consensus.cc:3060] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.352830 28889 rpcz_store.cc:276] 0320 15:27:30.709996 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.867123 (+157127us) service_pool.cc:225] Handling call
0320 15:27:30.868497 (+ 1374us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:30.868637 (+ 140us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:30.868709 (+ 72us) catalog_manager.cc:2824] Locking tablets
0320 15:27:30.927018 (+ 58309us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:31.098250 (+171232us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:31.098375 (+ 125us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:33.351771 (+2253396us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:33.352628 (+ 857us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:30.998151 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:30.998294 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:30.998303 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:30.998306 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:30.998443 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:30.998458 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:30.998465 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:30.998601 (+ 136us) tablet.cc:700] Row locks acquired
0320 15:27:30.998603 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:30.998642 (+ 39us) write_op.cc:270] Start()
0320 15:27:30.999221 (+ 579us) write_op.cc:276] Timestamp: P: 1774020450998597 usec, L: 0
0320 15:27:30.999226 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:30.999457 (+ 231us) log.cc:844] Serialized 725 byte log entry
0320 15:27:31.095469 (+ 96012us) op_driver.cc:464] REPLICATION: finished
0320 15:27:31.096702 (+ 1233us) write_op.cc:301] APPLY: starting
0320 15:27:31.096729 (+ 27us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:31.096788 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:31.096833 (+ 45us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:31.096920 (+ 87us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:31.096959 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:31.096967 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:31.097124 (+ 157us) log.cc:844] Serialized 29 byte log entry
0320 15:27:31.097808 (+ 684us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:31.097833 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:31.097842 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1135,"apply.run_cpu_time_us":590,"apply.run_wall_time_us":1228,"num_ops":2,"prepare.queue_time_us":70453,"prepare.run_cpu_time_us":957,"prepare.run_wall_time_us":7445,"raft.queue_time_us":60,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":96182}]]}
I20260320 15:27:33.354207 28733 tablet_bootstrap.cc:492] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:33.354422 28733 ts_tablet_manager.cc:1403] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.196s user 0.006s sys 0.008s
I20260320 15:27:33.356007 28733 raft_consensus.cc:359] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.356328 28733 raft_consensus.cc:385] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.356551 28733 raft_consensus.cc:740] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.356925 28733 consensus_queue.cc:260] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.357918 28733 ts_tablet_manager.cc:1434] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:33.367096 29445 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.316275 29245 raft_consensus.cc:515] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.371572 28411 tablet_service.cc:1511] Processing CreateTablet for tablet d04ab44039124f36ac60601098586b4e (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.376863 28418 tablet_service.cc:1511] Processing CreateTablet for tablet e779a274913f4caeaf82e3f180457830 (DEFAULT_TABLE table=default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.377946 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f0242a7968541f480367b45cda5d9cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:33.378014 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba3def9ab7641d2ad4a6d0538bf978a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.379133 28442 raft_consensus.cc:3060] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.382962 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d12bd6ae67f4600a0d54e7d5d502dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.369436 29445 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.388634 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0d80754dfd4bbd9817f568063511ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.395319 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.399039 29626 leader_election.cc:290] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.401474 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.403113 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.403695 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6498952e19c45dbacb1fa7af8ece8f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.409778 28408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2e17c67c544e38840c5dd8dec32e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.410841 29245 leader_election.cc:290] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.417368 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.417987 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:33.419709 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
I20260320 15:27:33.420696 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d04ab44039124f36ac60601098586b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.423645 28427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b16ffb28029e4b7ca37ce36efe9e0271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.420593 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f1a65661949258f521dc847e674cd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.426216 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet aafd1e123cd043e7828140d2443e047f
I20260320 15:27:33.427592 28892 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 38) took 2784 ms. Trace:
I20260320 15:27:33.429301 29553 raft_consensus.cc:2804] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.429920 29553 raft_consensus.cc:493] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.430074 29553 raft_consensus.cc:3060] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:33.430791 28906 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
W20260320 15:27:33.431164 28903 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d94b4a6d75804c3e947af4f588ef8f5a has no consensus state
I20260320 15:27:33.429740 28892 rpcz_store.cc:276] 0320 15:27:30.643173 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.644507 (+ 1334us) service_pool.cc:225] Handling call
0320 15:27:30.645122 (+ 615us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:30.645290 (+ 168us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:30.645359 (+ 69us) catalog_manager.cc:2824] Locking tablets
0320 15:27:30.922497 (+277138us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:31.034458 (+111961us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:31.034564 (+ 106us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:33.425981 (+2391417us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:33.427306 (+ 1325us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:30.923451 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:30.925297 (+ 1846us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:30.925304 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:30.925306 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:30.925427 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:30.925442 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:30.925447 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:30.925568 (+ 121us) tablet.cc:700] Row locks acquired
0320 15:27:30.925571 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:30.925612 (+ 41us) write_op.cc:270] Start()
0320 15:27:30.925656 (+ 44us) write_op.cc:276] Timestamp: P: 1774020450925570 usec, L: 0
0320 15:27:30.925660 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:30.992289 (+ 66629us) log.cc:844] Serialized 756 byte log entry
0320 15:27:30.992548 (+ 259us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:31.033766 (+ 41218us) op_driver.cc:464] REPLICATION: finished
0320 15:27:31.033875 (+ 109us) write_op.cc:301] APPLY: starting
0320 15:27:31.033897 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:31.033949 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:31.033952 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:31.034038 (+ 86us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:31.034078 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:31.034089 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:31.034190 (+ 101us) log.cc:844] Serialized 29 byte log entry
0320 15:27:31.034269 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:31.034295 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:31.034303 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":44,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":497,"num_ops":2,"prepare.queue_time_us":70,"prepare.run_cpu_time_us":2250,"prepare.run_wall_time_us":69200,"raft.queue_time_us":2225,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":325,"replication_time_us":108056,"spinlock_wait_cycles":152609664}]]}
I20260320 15:27:33.432711 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_8f15c883f7c74571a7717c5dcdfb4f6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:33.434335 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f15c883f7c74571a7717c5dcdfb4f6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:33.443701 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
I20260320 15:27:33.447978 29617 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:33.462193 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:33.462424 28524 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:33.447829 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.484337 28444 raft_consensus.cc:2468] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:33.486032 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.491612 28736 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:33.491832 28442 raft_consensus.cc:2468] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:33.493330 28383 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
W20260320 15:27:33.493552 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 50177.8us
I20260320 15:27:33.494342 28734 tablet_bootstrap.cc:654] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.498086 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:33.500643 28382 leader_election.cc:343] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:33.501138 28382 leader_election.cc:304] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:33.502061 29246 raft_consensus.cc:2749] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:33.502982 28736 ts_tablet_manager.cc:1403] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.348s user 0.013s sys 0.033s
I20260320 15:27:33.503175 28279 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.506304 28302 raft_consensus.cc:2468] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:33.520442 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 4c6f8c20b5904eafb9a62d14bb155c82 consensus state has no leader
I20260320 15:27:33.524385 28736 raft_consensus.cc:359] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.524706 28736 raft_consensus.cc:385] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.525593 28736 raft_consensus.cc:740] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.526206 28736 consensus_queue.cc:260] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.529318 28736 ts_tablet_manager.cc:1434] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260320 15:27:33.529727 28736 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:33.535676 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f94140a5e8b4a06890bca86456d64a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.542351 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:33.542827 29617 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.543381 29445 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.542398 28733 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:33.547240 29445 raft_consensus.cc:2749] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:33.548661 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.550616 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.552440 28736 tablet_bootstrap.cc:654] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.552523 28383 leader_election.cc:304] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:33.551595 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet aafd1e123cd043e7828140d2443e047f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aafd1e123cd043e7828140d2443e047f already in progress: creating tablet
I20260320 15:27:33.553659 28298 raft_consensus.cc:2468] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:33.561965 29245 raft_consensus.cc:2804] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.562922 28733 tablet_bootstrap.cc:654] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:33.562911 28382 leader_election.cc:343] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.565815 29245 raft_consensus.cc:493] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.565982 29245 raft_consensus.cc:3060] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.566928 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aba79ddde6e842dfb67597c5d9e371cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.568732 28587 raft_consensus.cc:3060] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.571754 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f3ee3c611d41b483bd35f868042d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.575055 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.579222 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50c1398534d84701ae7085a037f236ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.585446 28418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e779a274913f4caeaf82e3f180457830. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:33.597671 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet db81df6d3e9d49efb94c4f24de9db429 consensus state has no leader
I20260320 15:27:33.601500 29640 tablet_replica.cc:333] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:33.601917 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.604710 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.606932 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.608460 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 2f5beaa1bc1e4465be34c8e86f131559 consensus state has no leader
W20260320 15:27:33.609359 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fa54115d4df74bec85970057b5c3dd51 consensus state has no leader
W20260320 15:27:33.610055 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 5de834725bda4f249882914f1247ab62 consensus state has no leader
W20260320 15:27:33.610457 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 06cb44b9cc6445cf8324cf52190d6773 consensus state has no leader
W20260320 15:27:33.611290 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3c7a605bdcf14d2ab64ad078efd3d999 consensus state has no leader
W20260320 15:27:33.612206 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8d92e17cea044541968356ae993be1e7 consensus state has no leader
W20260320 15:27:33.612538 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet d78ab2f7e0d7413ea9049dba9cbd8633 consensus state has no leader
I20260320 15:27:33.612730 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.612862 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 1ccfd2d5e82448b99f1022954b98c6cc consensus state has no leader
I20260320 15:27:33.626003 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.626266 29640 raft_consensus.cc:2243] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:33.630750 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 07ff84dedc074dabb4e8e2d8460066e7 consensus state has no leader
I20260320 15:27:33.630944 29640 raft_consensus.cc:2272] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:33.631145 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 28c453eaf49c4171b0f93d3b26690d83 consensus state has no leader
W20260320 15:27:33.631376 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet b4cad900411e4c9686701aa99206dd23 consensus state has no leader
I20260320 15:27:33.636972 28588 raft_consensus.cc:3060] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.649331 29553 raft_consensus.cc:515] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.650123 29553 leader_election.cc:290] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:33.652768 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet ca5e0c215f574b8aafec8d5e2da9a399 consensus state has no leader
W20260320 15:27:33.653560 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 6ba738a76a424c18b36985a1977bcee4 consensus state has no leader
W20260320 15:27:33.654076 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ba4eb20eb0f1403282030727757031f7 consensus state has no leader
I20260320 15:27:33.639642 29640 ts_tablet_manager.cc:1916] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:33.664330 29245 raft_consensus.cc:515] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.667697 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.668852 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:33.669654 28382 leader_election.cc:343] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.669962 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.670327 28872 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet ca5e0c215f574b8aafec8d5e2da9a399 consensus state has no leader
I20260320 15:27:33.673575 29245 leader_election.cc:290] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:33.675864 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet f8f70142c3a449439a0a475bddafdd40 consensus state has no leader
I20260320 15:27:33.677526 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.683631 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.684311 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 40) took 3037 ms. Trace:
I20260320 15:27:33.684643 29599 raft_consensus.cc:493] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.684919 29245 raft_consensus.cc:493] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.684633 28906 rpcz_store.cc:276] 0320 15:27:30.646833 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.663175 (+ 16342us) service_pool.cc:225] Handling call
0320 15:27:30.663835 (+ 660us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:30.664099 (+ 264us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:30.664345 (+ 246us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:30.664425 (+ 80us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:30.699702 (+ 35277us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:33.684208 (+2984506us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:30.665432 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:30.665582 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:30.665591 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:30.665594 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:30.665727 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:30.665742 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:30.665747 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:30.665821 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:30.665824 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:30.665867 (+ 43us) write_op.cc:270] Start()
0320 15:27:30.665907 (+ 40us) write_op.cc:276] Timestamp: P: 1774020450665848 usec, L: 0
0320 15:27:30.665911 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:30.666139 (+ 228us) log.cc:844] Serialized 405 byte log entry
0320 15:27:30.695787 (+ 29648us) op_driver.cc:464] REPLICATION: finished
0320 15:27:30.696560 (+ 773us) write_op.cc:301] APPLY: starting
0320 15:27:30.696574 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:30.696612 (+ 38us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:30.696643 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:30.696688 (+ 45us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:30.696710 (+ 22us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:30.696717 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:30.696800 (+ 83us) log.cc:844] Serialized 23 byte log entry
0320 15:27:30.696849 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:30.696860 (+ 11us) write_op.cc:454] Released schema lock
0320 15:27:30.696865 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:30.696942 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:30.696948 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":739,"apply.run_cpu_time_us":375,"apply.run_wall_time_us":375,"num_ops":1,"prepare.queue_time_us":270,"prepare.run_cpu_time_us":865,"prepare.run_wall_time_us":1322,"raft.queue_time_us":82,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":29861}]]}
I20260320 15:27:33.685845 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_215c6d41f475486a94794ebbd60809c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:33.686698 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_215c6d41f475486a94794ebbd60809c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:33.687992 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 44) took 2978 ms. Trace:
I20260320 15:27:33.688163 28903 rpcz_store.cc:276] 0320 15:27:30.709055 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.709297 (+ 242us) service_pool.cc:225] Handling call
0320 15:27:30.730897 (+ 21600us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:30.731231 (+ 334us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:30.731483 (+ 252us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:30.737227 (+ 5744us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:30.737324 (+ 97us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:30.765448 (+ 28124us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:33.687930 (+2922482us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:30.738029 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:30.738198 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:30.738209 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:30.738211 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:30.738349 (+ 138us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:30.738369 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:30.738378 (+ 9us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:30.738460 (+ 82us) tablet.cc:700] Row locks acquired
0320 15:27:30.738462 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:30.738511 (+ 49us) write_op.cc:270] Start()
0320 15:27:30.738553 (+ 42us) write_op.cc:276] Timestamp: P: 1774020450738483 usec, L: 0
0320 15:27:30.738556 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:30.738789 (+ 233us) log.cc:844] Serialized 405 byte log entry
0320 15:27:30.764714 (+ 25925us) op_driver.cc:464] REPLICATION: finished
0320 15:27:30.764838 (+ 124us) write_op.cc:301] APPLY: starting
0320 15:27:30.764864 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:30.764930 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:30.764966 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:30.765035 (+ 69us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:30.765075 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:30.765083 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:30.765177 (+ 94us) log.cc:844] Serialized 23 byte log entry
0320 15:27:30.765263 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:30.765286 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:30.765295 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":48825600,"child_traces":[["op",{"apply.queue_time_us":61,"apply.run_cpu_time_us":531,"apply.run_wall_time_us":530,"num_ops":1,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":895,"prepare.run_wall_time_us":893,"raft.queue_time_us":20594,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":1156,"replication_time_us":26112}]]}
I20260320 15:27:33.689435 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c8c2dd201dc543b59a93b39cb64d5d83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:33.689759 29245 raft_consensus.cc:515] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:33.690228 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8c2dd201dc543b59a93b39cb64d5d83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:33.691602 29599 raft_consensus.cc:515] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.692291 29599 leader_election.cc:290] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:33.698632 29245 leader_election.cc:290] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.700783 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:33.701004 28583 raft_consensus.cc:2468] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:33.701335 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.701459 28382 leader_election.cc:304] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:33.701620 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.701905 29245 raft_consensus.cc:2804] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.702091 29245 raft_consensus.cc:493] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.702266 29245 raft_consensus.cc:3060] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.712260 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.713294 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.713497 28408 tablet_service.cc:1511] Processing CreateTablet for tablet 92ea5e924a734c8ba6dfb641359593a3 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.713739 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.713980 28408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ea5e924a734c8ba6dfb641359593a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.714170 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 50) took 2937 ms. Trace:
I20260320 15:27:33.714412 28892 rpcz_store.cc:276] 0320 15:27:30.777091 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.432091 (+2655000us) service_pool.cc:225] Handling call
0320 15:27:33.684182 (+252091us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.684871 (+ 689us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:33.713735 (+ 28864us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.713746 (+ 11us) write_op.cc:454] Released schema lock
0320 15:27:33.713868 (+ 122us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:33.714004 (+ 136us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:33.714121 (+ 117us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.686174 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.686352 (+ 178us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.686361 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:33.686366 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.686489 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.686508 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.686516 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.686622 (+ 106us) tablet.cc:700] Row locks acquired
0320 15:27:33.686626 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:33.686672 (+ 46us) write_op.cc:270] Start()
0320 15:27:33.686721 (+ 49us) write_op.cc:276] Timestamp: P: 1774020453686604 usec, L: 0
0320 15:27:33.686726 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.686960 (+ 234us) log.cc:844] Serialized 433 byte log entry
0320 15:27:33.699878 (+ 12918us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.700065 (+ 187us) write_op.cc:301] APPLY: starting
0320 15:27:33.700084 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.700137 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.700173 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.700326 (+ 153us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.700376 (+ 50us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:33.700388 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:33.700484 (+ 96us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.700558 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.700583 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:33.700591 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":591,"apply.run_wall_time_us":591,"num_ops":2,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":919,"prepare.run_wall_time_us":920,"raft.queue_time_us":11997,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":333,"replication_time_us":13119}]]}
I20260320 15:27:33.715281 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.715726 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.716213 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.716487 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_fa221247f0074abdbafb6ec471027c6a" } modify_external_catalogs: true
I20260320 15:27:33.717689 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.719041 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.719627 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.724148 28411 tablet_service.cc:1511] Processing CreateTablet for tablet d94b4a6d75804c3e947af4f588ef8f5a (DEFAULT_TABLE table=default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.724741 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94b4a6d75804c3e947af4f588ef8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.725562 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_fa221247f0074abdbafb6ec471027c6a" } modify_external_catalogs: true
I20260320 15:27:33.727286 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.729087 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 51) took 2950 ms. Trace:
W20260320 15:27:33.725075 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ce8595157679444b959b569dbb7d8df0 consensus state has no leader
I20260320 15:27:33.729216 28906 rpcz_store.cc:276] 0320 15:27:30.778139 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.685591 (+2907452us) service_pool.cc:225] Handling call
0320 15:27:33.686811 (+ 1220us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.687315 (+ 504us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:33.728757 (+ 41442us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:33.728898 (+ 141us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:33.729006 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.694271 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.694430 (+ 159us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.694440 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:33.694445 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.694567 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.694584 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.694591 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.694706 (+ 115us) tablet.cc:700] Row locks acquired
0320 15:27:33.694710 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:33.694755 (+ 45us) write_op.cc:270] Start()
0320 15:27:33.694801 (+ 46us) write_op.cc:276] Timestamp: P: 1774020453694687 usec, L: 0
0320 15:27:33.694807 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.695059 (+ 252us) log.cc:844] Serialized 433 byte log entry
0320 15:27:33.705898 (+ 10839us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.706227 (+ 329us) write_op.cc:301] APPLY: starting
0320 15:27:33.706263 (+ 36us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.706326 (+ 63us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.706380 (+ 54us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.706520 (+ 140us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.706562 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:33.706568 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:33.706664 (+ 96us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.706733 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.706758 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:33.706765 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:33.706853 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.706857 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":251,"apply.run_cpu_time_us":629,"apply.run_wall_time_us":628,"num_ops":2,"prepare.queue_time_us":6387,"prepare.run_cpu_time_us":923,"prepare.run_wall_time_us":921,"raft.queue_time_us":778,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":11046}]]}
I20260320 15:27:33.730317 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_4e32eac691b748468ec7ad07338699b7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:33.722896 28903 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 52) took 2944 ms. Trace:
W20260320 15:27:33.731271 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e32eac691b748468ec7ad07338699b7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:33.731290 28903 rpcz_store.cc:276] 0320 15:27:30.778284 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.689173 (+2910889us) service_pool.cc:225] Handling call
0320 15:27:33.690397 (+ 1224us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.690908 (+ 511us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:33.722538 (+ 31630us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:33.722674 (+ 136us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:33.722810 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.695215 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.695316 (+ 101us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.695321 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:33.695323 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:33.695439 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.695453 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.695461 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.695580 (+ 119us) tablet.cc:700] Row locks acquired
0320 15:27:33.695585 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:33.695619 (+ 34us) write_op.cc:270] Start()
0320 15:27:33.695652 (+ 33us) write_op.cc:276] Timestamp: P: 1774020453695554 usec, L: 0
0320 15:27:33.695656 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.695790 (+ 134us) log.cc:844] Serialized 433 byte log entry
0320 15:27:33.705985 (+ 10195us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.706566 (+ 581us) write_op.cc:301] APPLY: starting
0320 15:27:33.706578 (+ 12us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.706612 (+ 34us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.706678 (+ 66us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.706819 (+ 141us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.706845 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:33.706851 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:33.706924 (+ 73us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.707421 (+ 497us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.707439 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:33.707445 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:33.707520 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.707523 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":520,"apply.run_cpu_time_us":535,"apply.run_wall_time_us":940,"mutex_wait_us":411,"num_ops":2,"prepare.queue_time_us":3831,"prepare.run_cpu_time_us":652,"prepare.run_wall_time_us":651,"raft.queue_time_us":105,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":10305,"thread_start_us":1149,"threads_started":1}]]}
I20260320 15:27:33.742816 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.743608 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.743486 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.744127 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_401dcb9cf68646098dd34f567b99c2f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:33.745289 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_401dcb9cf68646098dd34f567b99c2f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:33.747745 28300 raft_consensus.cc:3060] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.747828 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 92ea5e924a734c8ba6dfb641359593a3 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:33.747651 28281 tablet_service.cc:1511] Processing CreateTablet for tablet d94b4a6d75804c3e947af4f588ef8f5a (DEFAULT_TABLE table=default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:33.751327 28900 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0e3f6d9a528946908891f77f951d23dd consensus state has no leader
I20260320 15:27:33.751791 29245 raft_consensus.cc:515] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.752544 29245 leader_election.cc:290] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.761245 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:33.761545 28584 raft_consensus.cc:3060] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.773662 28301 raft_consensus.cc:2468] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:33.776543 28302 raft_consensus.cc:2468] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:33.776636 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 342295us
I20260320 15:27:33.779578 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.780042 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f1a65661949258f521dc847e674cd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:33.780172 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.780269 28443 raft_consensus.cc:3060] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.782783 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ea5e924a734c8ba6dfb641359593a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.784466 28734 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:33.784744 28734 ts_tablet_manager.cc:1403] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.643s user 0.007s sys 0.013s
I20260320 15:27:33.785939 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.787138 28734 raft_consensus.cc:359] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.787381 28734 raft_consensus.cc:385] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:33.787453 28734 raft_consensus.cc:740] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:33.787797 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.789923 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.790246 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.784315 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.795465 28298 raft_consensus.cc:3060] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.796330 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94b4a6d75804c3e947af4f588ef8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:33.807971 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.815128 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.822809 28410 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.823232 29643 tablet_replica.cc:333] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:33.823519 29643 raft_consensus.cc:2243] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:33.824025 29643 raft_consensus.cc:2272] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:33.824901 29643 ts_tablet_manager.cc:1916] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:33.827670 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.828773 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:33.828827 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 371717us
I20260320 15:27:33.829741 28734 consensus_queue.cc:260] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.830061 29245 raft_consensus.cc:493] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.830230 29245 raft_consensus.cc:515] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.830476 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:33.831293 29245 leader_election.cc:290] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.831563 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.831708 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442ca186c9d4449dade39271dec5c608" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.831838 28299 raft_consensus.cc:3060] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.831923 28304 raft_consensus.cc:2468] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:33.831995 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.832602 28383 leader_election.cc:304] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:33.832957 29245 raft_consensus.cc:2804] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:33.833129 29245 raft_consensus.cc:493] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:33.833285 29245 raft_consensus.cc:3060] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.833640 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2775196c8e04dcc80c7e089b57579fd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:33.839690 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:33.840052 28446 raft_consensus.cc:2393] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:33.840909 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.845587 28903 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 55) took 3001 ms. Trace:
I20260320 15:27:33.846979 28443 raft_consensus.cc:2468] T e13f1a65661949258f521dc847e674cd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:33.848167 28903 rpcz_store.cc:276] 0320 15:27:30.844157 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.743831 (+2899674us) service_pool.cc:225] Handling call
0320 15:27:33.745507 (+ 1676us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.746098 (+ 591us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:33.771769 (+ 25671us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.771779 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:33.771896 (+ 117us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:33.845363 (+ 73467us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:33.845502 (+ 139us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.758903 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.759078 (+ 175us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.759088 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:33.759093 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.759223 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.759240 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.759248 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.759367 (+ 119us) tablet.cc:700] Row locks acquired
0320 15:27:33.759372 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:33.759417 (+ 45us) write_op.cc:270] Start()
0320 15:27:33.759463 (+ 46us) write_op.cc:276] Timestamp: P: 1774020453759343 usec, L: 0
0320 15:27:33.759469 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.759864 (+ 395us) log.cc:844] Serialized 433 byte log entry
0320 15:27:33.766374 (+ 6510us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.766611 (+ 237us) write_op.cc:301] APPLY: starting
0320 15:27:33.766646 (+ 35us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.766712 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.766766 (+ 54us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.766923 (+ 157us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.766977 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:33.766989 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:33.767110 (+ 121us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.767201 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.767247 (+ 46us) write_op.cc:454] Released schema lock
0320 15:27:33.767257 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":151,"apply.run_cpu_time_us":719,"apply.run_wall_time_us":719,"num_ops":2,"prepare.queue_time_us":12111,"prepare.run_cpu_time_us":1072,"prepare.run_wall_time_us":1111,"raft.queue_time_us":1915,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":338,"replication_time_us":6842}]]}
I20260320 15:27:33.852425 29643 ts_tablet_manager.cc:1929] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:33.852674 29643 log.cc:1199] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/a2775196c8e04dcc80c7e089b57579fd
W20260320 15:27:33.853164 28900 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 90) took 3278 ms (client timeout 1000 ms). Trace:
I20260320 15:27:33.853230 29643 ts_tablet_manager.cc:1950] T a2775196c8e04dcc80c7e089b57579fd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:33.846035 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 54) took 3001 ms. Trace:
W20260320 15:27:33.853303 28900 rpcz_store.cc:269] 0320 15:27:30.574321 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.589691 (+ 15370us) service_pool.cc:225] Handling call
0320 15:27:30.630605 (+ 40914us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:33.853094 (+3222489us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:31.835653 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:31.835832 (+ 179us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:31.835844 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:31.835848 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:31.836021 (+ 173us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:31.836044 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:31.836051 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:31.837480 (+ 1429us) tablet.cc:700] Row locks acquired
0320 15:27:31.837485 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:31.837540 (+ 55us) write_op.cc:270] Start()
0320 15:27:31.837595 (+ 55us) write_op.cc:276] Timestamp: P: 1774020451837501 usec, L: 0
0320 15:27:31.837601 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:31.837942 (+ 341us) log.cc:844] Serialized 901 byte log entry
0320 15:27:32.583249 (+745307us) op_driver.cc:464] REPLICATION: finished
0320 15:27:32.611667 (+ 28418us) write_op.cc:301] APPLY: starting
0320 15:27:32.611693 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:32.611750 (+ 57us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:32.611795 (+ 45us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:32.611908 (+ 113us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:32.611949 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:32.611956 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:32.612083 (+ 127us) log.cc:844] Serialized 29 byte log entry
0320 15:27:32.612458 (+ 375us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.612492 (+ 34us) write_op.cc:454] Released schema lock
0320 15:27:32.612502 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":100382,"reported_tablets":152,"spinlock_wait_cycles":74828544,"child_traces":[["op",{"apply.queue_time_us":28289,"apply.run_cpu_time_us":705,"apply.run_wall_time_us":953,"num_ops":2,"prepare.queue_time_us":698,"prepare.run_cpu_time_us":1376,"prepare.run_wall_time_us":30005,"raft.queue_time_us":359,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":389,"replication_time_us":745603,"thread_start_us":41645,"threads_started":2}]]}
I20260320 15:27:33.853626 28906 rpcz_store.cc:276] 0320 15:27:30.844062 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.729894 (+2885832us) service_pool.cc:225] Handling call
0320 15:27:33.731415 (+ 1521us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.732055 (+ 640us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:33.824044 (+ 91989us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:33.845845 (+ 21801us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:33.845969 (+ 124us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.732985 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.733154 (+ 169us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.733163 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:33.733167 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:33.733317 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.733340 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.733349 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.733459 (+ 110us) tablet.cc:700] Row locks acquired
0320 15:27:33.733463 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:33.733616 (+ 153us) write_op.cc:270] Start()
0320 15:27:33.733681 (+ 65us) write_op.cc:276] Timestamp: P: 1774020453733440 usec, L: 0
0320 15:27:33.733688 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.733963 (+ 275us) log.cc:844] Serialized 433 byte log entry
0320 15:27:33.739602 (+ 5639us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.739704 (+ 102us) write_op.cc:301] APPLY: starting
0320 15:27:33.739728 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.739781 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.739810 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.739934 (+ 124us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.739971 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:33.739978 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:33.740068 (+ 90us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.740168 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.740205 (+ 37us) write_op.cc:454] Released schema lock
0320 15:27:33.740213 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:33.740342 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.740346 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":33,"apply.run_cpu_time_us":628,"apply.run_wall_time_us":628,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":1145,"prepare.run_wall_time_us":1622,"raft.queue_time_us":526,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":374,"replication_time_us":5864}]]}
I20260320 15:27:33.859470 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.866117 28524 leader_election.cc:304] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.869690 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.870462 29640 ts_tablet_manager.cc:1929] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:33.871246 28524 leader_election.cc:304] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:33.872123 29617 consensus_queue.cc:1048] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:33.874017 28524 leader_election.cc:343] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.875296 29640 log.cc:1199] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:33.879423 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.880307 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db81df6d3e9d49efb94c4f24de9db429" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:33.883020 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
W20260320 15:27:33.883419 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet aafd1e123cd043e7828140d2443e047f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aafd1e123cd043e7828140d2443e047f already in progress: creating tablet
I20260320 15:27:33.882268 28240 leader_election.cc:304] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:33.900591 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.904170 29599 raft_consensus.cc:2804] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:33.895922 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442ca186c9d4449dade39271dec5c608" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:33.906809 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.907508 28382 leader_election.cc:343] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.909062 29599 raft_consensus.cc:697] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:33.910292 29678 tablet_replica.cc:333] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:33.916023 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.918335 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.918852 29706 raft_consensus.cc:2804] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:33.919094 29706 raft_consensus.cc:697] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:33.920403 29678 raft_consensus.cc:2243] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:33.920959 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.921377 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.921928 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet a2775196c8e04dcc80c7e089b57579fd (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]) successfully deleted
I20260320 15:27:33.921401 29643 tablet_replica.cc:333] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:33.922396 29643 raft_consensus.cc:2243] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:33.922448 29678 raft_consensus.cc:2272] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:33.922284 29706 consensus_queue.cc:237] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.922601 29643 raft_consensus.cc:2272] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:33.923192 29678 ts_tablet_manager.cc:1916] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:33.923897 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.924245 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.924753 29643 ts_tablet_manager.cc:1916] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:33.925213 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.933856 29677 raft_consensus.cc:493] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:33.937870 28736 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:33.938220 28736 ts_tablet_manager.cc:1403] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.409s user 0.033s sys 0.082s
I20260320 15:27:33.938282 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.939836 29727 tablet.cc:1723] T d56de00174db4e2d8b1c7215038a04aa P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260320 15:27:33.940593 28524 leader_election.cc:343] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:33.941540 29676 raft_consensus.cc:2804] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:33.941730 29676 raft_consensus.cc:697] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:33.940172 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.944415 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.944700 28902 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 3f4a2c8ebc374de9bd8078a757c6be57 consensus state has no leader
I20260320 15:27:33.946430 28733 tablet_bootstrap.cc:492] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:33.945703 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 66204.1us
I20260320 15:27:33.948870 29245 raft_consensus.cc:515] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.949123 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.949756 29245 leader_election.cc:290] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:33.950199 28298 raft_consensus.cc:2468] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:33.950851 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.952880 28300 raft_consensus.cc:2468] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:33.953814 28734 ts_tablet_manager.cc:1434] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.169s user 0.004s sys 0.000s
I20260320 15:27:33.955749 28383 leader_election.cc:304] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:33.955744 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442ca186c9d4449dade39271dec5c608" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:33.956066 28298 raft_consensus.cc:3060] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:33.956561 28383 leader_election.cc:304] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:33.957547 28299 raft_consensus.cc:2468] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:33.958671 29245 raft_consensus.cc:2804] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:33.958873 29245 raft_consensus.cc:697] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:33.959043 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.959247 29245 consensus_queue.cc:237] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.959447 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.958987 29677 raft_consensus.cc:515] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.959751 29626 raft_consensus.cc:2804] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:33.961163 28734 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:33.960951 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_14baef3186e14a0ab4705c1cdaff1717"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:33.962047 29626 raft_consensus.cc:697] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:33.963482 29626 consensus_queue.cc:237] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.971346 28734 tablet_bootstrap.cc:654] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:33.972174 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.974985 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.975572 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.975910 29640 ts_tablet_manager.cc:1950] T b8f3ee3c611d41b483bd35f868042d05 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:33.979688 28902 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 008811c90aeb4ca4a4ad5c0190a60dcc consensus state has no leader
W20260320 15:27:33.980139 28902 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 57cbefe3a19044f78b64db951df5f7c1 consensus state has no leader
I20260320 15:27:33.981837 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.981879 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:33.983263 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 91) took 1793 ms (client timeout 1000 ms). Trace:
I20260320 15:27:33.983628 28733 ts_tablet_manager.cc:1403] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.507s user 0.003s sys 0.015s
I20260320 15:27:33.983979 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.984216 29599 consensus_queue.cc:237] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.984805 29733 tablet.cc:1723] T d56de00174db4e2d8b1c7215038a04aa P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:33.985616 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.986191 29676 consensus_queue.cc:237] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:33.987720 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:33.989991 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:33.991779 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:33.993323 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:33.993957 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14baef3186e14a0ab4705c1cdaff1717 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:34.001052 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.001302 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.002955 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.002616 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:34.005345 28862 consensus_peers.cc:597] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 -> Peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Couldn't send request to peer 07e8bb8819eb4cb4892e0cba1f758892. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260320 15:27:34.006141 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:34.007004 28734 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.007174 28734 ts_tablet_manager.cc:1403] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.003s
I20260320 15:27:34.008280 28734 raft_consensus.cc:359] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.008498 28734 raft_consensus.cc:385] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.008607 28734 raft_consensus.cc:740] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.008888 28734 consensus_queue.cc:260] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.009507 28734 ts_tablet_manager.cc:1434] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:34.009862 28734 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.011801 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.012624 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.013962 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.013777 28736 raft_consensus.cc:359] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.015270 28734 tablet_bootstrap.cc:654] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.016275 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.019320 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.020881 29744 tablet.cc:1723] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.021373 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.022125 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.022363 28736 raft_consensus.cc:385] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.022488 28736 raft_consensus.cc:740] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.022771 28736 consensus_queue.cc:260] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.024176 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.025871 28736 ts_tablet_manager.cc:1434] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.087s user 0.001s sys 0.002s
I20260320 15:27:34.026968 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.029080 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.031092 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.031245 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.031347 28734 tablet_bootstrap.cc:492] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.031558 28734 ts_tablet_manager.cc:1403] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.005s
I20260320 15:27:34.031919 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.032274 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.032675 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd with delete_type TABLET_DATA_DELETED (replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.032881 28734 raft_consensus.cc:359] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.033028 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.033130 28734 raft_consensus.cc:385] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.033257 28734 raft_consensus.cc:740] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:34.033299 28524 leader_election.cc:343] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.033581 28734 consensus_queue.cc:260] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:34.033835 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 52062.6us
I20260320 15:27:34.034232 28734 ts_tablet_manager.cc:1434] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:34.034574 28734 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:34.037011 28524 leader_election.cc:343] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:34.037354 28524 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: STOPPED
I20260320 15:27:34.037554 28524 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.038005 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.038444 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.035590 28733 raft_consensus.cc:359] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.038786 28733 raft_consensus.cc:385] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.038940 28733 raft_consensus.cc:740] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.039317 28733 consensus_queue.cc:260] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.039865 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.040060 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.040261 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.040398 28736 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.040833 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.042069 29677 leader_election.cc:290] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.047370 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "442ca186c9d4449dade39271dec5c608" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:34.050135 28734 tablet_bootstrap.cc:654] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.042367 28736 tablet_bootstrap.cc:654] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:34.053107 28240 leader_election.cc:343] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.055065 29643 ts_tablet_manager.cc:1929] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.055250 29643 log.cc:1199] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:34.058024 29643 ts_tablet_manager.cc:1950] T 9aa2d48a54dd403c82d7f3a64db5aeab P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:34.058449 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.058703 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.061264 28298 raft_consensus.cc:2468] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:34.053534 29752 tablet.cc:1723] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.063652 28383 leader_election.cc:304] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.064098 29246 raft_consensus.cc:2804] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:34.064265 29246 raft_consensus.cc:697] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:34.064611 29246 consensus_queue.cc:237] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.066643 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.042518 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.069245 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.069281 29742 tablet.cc:1723] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.073288 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.051019 29733 tablet.cc:1723] T 4b4040a0214a46b580b2ac2de59243ce P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.058622 28584 raft_consensus.cc:2468] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:34.061061 29640 tablet_replica.cc:333] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:34.071771 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.072075 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.072325 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.072623 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.072906 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.073158 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.073565 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.075845 28734 tablet_bootstrap.cc:492] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.076022 28734 ts_tablet_manager.cc:1403] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.003s
I20260320 15:27:34.076972 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.077031 28734 raft_consensus.cc:359] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.077234 28734 raft_consensus.cc:385] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.077355 28734 raft_consensus.cc:740] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.077663 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.078336 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.079164 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.079614 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca with delete_type TABLET_DATA_DELETED (replaced by tablet 9dd293a68f894cbfa122d53918816c28) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.079988 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.080626 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.081187 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.081797 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.082116 28734 consensus_queue.cc:260] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.082340 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.082894 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.083587 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.084244 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.082806 28734 ts_tablet_manager.cc:1434] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:34.087571 28734 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.089141 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.092209 29640 raft_consensus.cc:2243] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:34.092236 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.095342 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.095646 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.095734 29640 raft_consensus.cc:2272] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:34.096621 29676 raft_consensus.cc:2749] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.096948 28583 raft_consensus.cc:3060] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.097560 28734 tablet_bootstrap.cc:654] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.097888 29640 ts_tablet_manager.cc:1916] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:34.102876 29754 tablet.cc:1723] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.104283 29751 tablet.cc:1723] T 4b4040a0214a46b580b2ac2de59243ce P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260320 15:27:34.100131 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 43767.5us
W20260320 15:27:33.983412 28902 rpcz_store.cc:269] 0320 15:27:32.189888 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:32.922644 (+732756us) service_pool.cc:225] Handling call
0320 15:27:33.242912 (+320268us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.242925 (+ 13us) write_op.cc:454] Released schema lock
0320 15:27:33.983183 (+740258us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.231656 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.231831 (+ 175us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.231841 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:33.231846 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.231966 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.231983 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.232013 (+ 30us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:33.232084 (+ 71us) tablet.cc:700] Row locks acquired
0320 15:27:33.232087 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:33.232136 (+ 49us) write_op.cc:270] Start()
0320 15:27:33.232180 (+ 44us) write_op.cc:276] Timestamp: P: 1774020453232074 usec, L: 0
0320 15:27:33.232184 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.232437 (+ 253us) log.cc:844] Serialized 527 byte log entry
0320 15:27:33.237942 (+ 5505us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.238122 (+ 180us) write_op.cc:301] APPLY: starting
0320 15:27:33.238142 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.238193 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.238237 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.238311 (+ 74us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.238357 (+ 46us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:33.238364 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:33.238469 (+ 105us) log.cc:844] Serialized 23 byte log entry
0320 15:27:33.238562 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.238587 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:33.238596 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":105805,"reported_tablets":164,"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":542,"apply.run_wall_time_us":541,"num_ops":1,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":922,"prepare.run_wall_time_us":922,"raft.queue_time_us":2055,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":673,"replication_time_us":5716}]]}
I20260320 15:27:34.113765 28902 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_923e214b36464c938d8b5be225e0a94b" } new_table_name: "default.Table_b2fb7e79b74940ce87fe158e71ae31db" modify_external_catalogs: true
I20260320 15:27:34.119688 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.124213 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.124547 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.124667 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.125458 29676 raft_consensus.cc:493] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.125715 29676 raft_consensus.cc:515] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.127094 28733 ts_tablet_manager.cc:1434] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.114s user 0.000s sys 0.003s
I20260320 15:27:34.127812 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.124255 29753 tablet.cc:1723] T 8f6f683638874e90b0d3c8969c702a5c P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:34.105616 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.106294 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.130144 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.106948 28734 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.130906 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.131232 28734 ts_tablet_manager.cc:1403] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.000s
I20260320 15:27:34.131892 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.132360 28734 raft_consensus.cc:359] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.132616 28734 raft_consensus.cc:385] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.132710 28734 raft_consensus.cc:740] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.133008 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.135192 28734 consensus_queue.cc:260] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.136626 28734 ts_tablet_manager.cc:1434] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:34.138096 28734 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:34.140393 29740 log.cc:927] Time spent T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260320 15:27:34.141359 29706 raft_consensus.cc:493] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.142655 28734 tablet_bootstrap.cc:654] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.144596 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.145092 28733 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.134351 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.146116 29706 raft_consensus.cc:515] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.146634 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.146915 28588 raft_consensus.cc:3060] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.147004 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.147346 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.147729 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.148212 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.148895 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.150089 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.150048 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.151065 29676 leader_election.cc:290] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.151683 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:34.153496 28240 leader_election.cc:343] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.157157 28734 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.157416 28734 ts_tablet_manager.cc:1403] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260320 15:27:34.155004 29706 leader_election.cc:290] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:34.159691 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.161219 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.162468 28734 raft_consensus.cc:359] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.162750 28734 raft_consensus.cc:385] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.162855 28734 raft_consensus.cc:740] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.163131 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.163201 28734 consensus_queue.cc:260] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.163765 28270 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.163841 28283 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.164053 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.164108 29678 ts_tablet_manager.cc:1929] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.164269 29678 log.cc:1199] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:34.165470 28734 ts_tablet_manager.cc:1434] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260320 15:27:34.166158 28734 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.166821 29643 tablet_replica.cc:333] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:34.167114 29643 raft_consensus.cc:2243] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:34.167389 29643 raft_consensus.cc:2272] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:34.167461 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.168743 29678 ts_tablet_manager.cc:1950] T 9aa2d48a54dd403c82d7f3a64db5aeab P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:34.168785 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.168936 29643 ts_tablet_manager.cc:1916] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:34.169389 29678 tablet_replica.cc:333] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:34.170722 29727 tablet.cc:1723] T 5b263f9a5df2477e8dc4cd52e5834991 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.173882 28734 tablet_bootstrap.cc:654] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.174365 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.174531 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.175480 28524 leader_election.cc:304] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.176306 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.176689 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.176908 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.177078 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.177248 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.177450 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.177872 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.177819 29762 tablet.cc:1723] T 5b263f9a5df2477e8dc4cd52e5834991 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.178195 29626 consensus_queue.cc:1048] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.180124 29733 tablet.cc:1723] T 9dd293a68f894cbfa122d53918816c28 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.180969 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.182332 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.183117 29678 raft_consensus.cc:2243] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:34.185113 29676 raft_consensus.cc:2749] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.186587 29678 raft_consensus.cc:2272] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:34.188724 29678 ts_tablet_manager.cc:1916] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:34.188767 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:34.190934 28872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.60:42563, user_credentials={real_user=slave}} blocked reactor thread for 63934.6us
I20260320 15:27:34.192593 28548 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:51046 (request call id 173) took 1056 ms. Trace:
I20260320 15:27:34.193413 28524 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.193600 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.189990 29761 tablet.cc:1723] T 4b4040a0214a46b580b2ac2de59243ce P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.194461 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.194561 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.194782 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.194890 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.195135 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.195329 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.195567 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.195672 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.195819 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.196053 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.196352 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.196590 28410 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.197237 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.197453 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.197556 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.197664 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.197888 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.198094 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.198336 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.198597 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.198824 28736 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:34.199038 28736 ts_tablet_manager.cc:1403] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.171s user 0.011s sys 0.015s
I20260320 15:27:34.193404 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.201761 28736 raft_consensus.cc:359] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.202086 28736 raft_consensus.cc:385] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.202255 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.202888 28736 raft_consensus.cc:740] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.203076 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.203962 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.204114 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.204372 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.204999 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.206326 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.206710 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.205873 28736 consensus_queue.cc:260] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.192806 28548 rpcz_store.cc:276] 0320 15:27:33.129681 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.131412 (+ 1731us) service_pool.cc:225] Handling call
0320 15:27:33.389879 (+258467us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x616000113b54. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965cdfe8c0 00007f9660cb5eaf 00007f965b0d1c36 00007f965b0d1965 00007f9657eeee3e 00007f9657eedd1e 00007f9657ef3f55 00007f9657ef487f 00007f9657ef46a8 0000000000a8b58c
0320 15:27:33.560119 (+170240us) spinlock_profiling.cc:243] Waited 885 us on lock 0x6160001d0254. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965cdfe8c0 00007f9660cb5eaf 00007f965b0d1c36 00007f965b0d1965 00007f9657eeee3e 00007f9657eedd1e 00007f9657ef3f55 00007f9657ef487f 00007f9657ef46a8 0000000000a8b58c
0320 15:27:33.944029 (+383910us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x616000220354. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965cdfe8c0 00007f9660cb5eaf 00007f965b0d1c36 00007f965b0d1965 00007f9657eeee3e 00007f9657eedd1e 00007f9657ef3f55 00007f9657ef487f 00007f9657ef46a8 0000000000a8b58c
0320 15:27:34.076373 (+132344us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x616000220954. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965cdfe8c0 00007f9660cb5eaf 00007f965b0d1c36 00007f965b0d1965 00007f9657eeee3e 00007f9657eedd1e 00007f9657ef3f55 00007f9657ef487f 00007f9657ef46a8 0000000000a8b58c
0320 15:27:34.186527 (+110154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":577807616}
I20260320 15:27:34.211640 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.214042 29676 raft_consensus.cc:2804] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:34.216169 29676 raft_consensus.cc:697] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:34.217818 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.218644 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.218669 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.218904 29676 consensus_queue.cc:237] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.220230 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.220557 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.220953 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.220989 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.221416 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.221835 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.222229 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:34.221670 28890 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 98) took 3546 ms (client timeout 1000 ms). Trace:
I20260320 15:27:34.222802 28733 tablet_bootstrap.cc:654] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.222827 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.223356 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.223851 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.224262 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.224478 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 92ea5e924a734c8ba6dfb641359593a3 (DEFAULT_TABLE table=default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.224681 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.224980 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.225049 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92ea5e924a734c8ba6dfb641359593a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.225322 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.225765 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.226078 28559 tablet_service.cc:1511] Processing CreateTablet for tablet d94b4a6d75804c3e947af4f588ef8f5a (DEFAULT_TABLE table=default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.226706 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d94b4a6d75804c3e947af4f588ef8f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.229063 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.229324 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.229441 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.236837 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.237071 29673 consensus_queue.cc:1048] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260320 15:27:34.243773 28736 ts_tablet_manager.cc:1434] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.041s user 0.003s sys 0.000s
I20260320 15:27:34.244204 28736 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.245780 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9aa2d48a54dd403c82d7f3a64db5aeab (table default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]) successfully deleted
W20260320 15:27:34.245975 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 53272.4us
I20260320 15:27:34.248318 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 825f7e3fdc004fe1a3bcca6deb20cccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.249842 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e13f1a65661949258f521dc847e674cd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:34.250030 28584 raft_consensus.cc:3060] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:34.251230 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 112) took 3871 ms (client timeout 1000 ms). Trace:
I20260320 15:27:34.253185 29744 tablet.cc:1723] T 8f6f683638874e90b0d3c8969c702a5c P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:34.253790 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.253916 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.254077 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet d547c032332d4d109222b4743fa28561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.254153 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.254285 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:34.254384 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet aafd1e123cd043e7828140d2443e047f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aafd1e123cd043e7828140d2443e047f already in progress: creating tablet
I20260320 15:27:34.254462 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.254979 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b8f3ee3c611d41b483bd35f868042d05 (table default.Table_78dd3a47806c44f186bb967be20e8d9f [id=79ab13b2c3fa42b4b744700487fd8b8d]) successfully deleted
I20260320 15:27:34.257330 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.257558 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.257566 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 56) took 3413 ms. Trace:
I20260320 15:27:34.257666 28900 rpcz_store.cc:276] 0320 15:27:30.844271 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.960421 (+3116150us) service_pool.cc:225] Handling call
0320 15:27:33.994303 (+ 33882us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:33.995094 (+ 791us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:34.074007 (+ 78913us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.074017 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:34.074164 (+ 147us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:34.257372 (+183208us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:34.257497 (+ 125us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.996251 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.996439 (+ 188us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.996451 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:33.996456 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.996574 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.996591 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.996597 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.996717 (+ 120us) tablet.cc:700] Row locks acquired
0320 15:27:33.996722 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:33.996784 (+ 62us) write_op.cc:270] Start()
0320 15:27:33.996837 (+ 53us) write_op.cc:276] Timestamp: P: 1774020453996738 usec, L: 0
0320 15:27:33.996842 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.997125 (+ 283us) log.cc:844] Serialized 433 byte log entry
0320 15:27:34.002623 (+ 5498us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.002788 (+ 165us) write_op.cc:301] APPLY: starting
0320 15:27:34.002809 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.002862 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.002911 (+ 49us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.003034 (+ 123us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.003073 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:34.003080 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:34.003188 (+ 108us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.003267 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.003293 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:34.003310 (+ 17us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":99,"apply.run_cpu_time_us":589,"apply.run_wall_time_us":588,"num_ops":2,"prepare.queue_time_us":281,"prepare.run_cpu_time_us":1055,"prepare.run_wall_time_us":1055,"raft.queue_time_us":581,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":317,"replication_time_us":5731}]]}
I20260320 15:27:34.258755 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_41024969454c4959a48da8f41b8531c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:34.258968 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9aa2d48a54dd403c82d7f3a64db5aeab (table default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]) successfully deleted
I20260320 15:27:34.260483 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ba738a76a424c18b36985a1977bcee4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.261333 28302 raft_consensus.cc:2393] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:34.263173 29762 tablet.cc:1723] T 8f6f683638874e90b0d3c8969c702a5c P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:34.265058 28734 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.265213 28734 ts_tablet_manager.cc:1403] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.099s user 0.007s sys 0.008s
I20260320 15:27:34.266070 28734 raft_consensus.cc:359] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.266266 28734 raft_consensus.cc:385] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.266330 28734 raft_consensus.cc:740] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:34.251358 28905 rpcz_store.cc:269] 0320 15:27:30.377473 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.552388 (+174915us) service_pool.cc:225] Handling call
0320 15:27:30.624971 (+ 72583us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:32.912882 (+2287911us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.912889 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:34.248499 (+1335610us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:32.904615 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:32.904758 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:32.904767 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:32.904770 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:32.904950 (+ 180us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:32.904968 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:32.904973 (+ 5us) tablet.cc:684] Acquiring locks for 3 operations
0320 15:27:32.905070 (+ 97us) tablet.cc:700] Row locks acquired
0320 15:27:32.905073 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:32.905110 (+ 37us) write_op.cc:270] Start()
0320 15:27:32.905148 (+ 38us) write_op.cc:276] Timestamp: P: 1774020452905081 usec, L: 0
0320 15:27:32.905152 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:32.905349 (+ 197us) log.cc:844] Serialized 1275 byte log entry
0320 15:27:32.909725 (+ 4376us) op_driver.cc:464] REPLICATION: finished
0320 15:27:32.909866 (+ 141us) write_op.cc:301] APPLY: starting
0320 15:27:32.909886 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:32.909959 (+ 73us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:32.910008 (+ 49us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:32.910103 (+ 95us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:32.910142 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0320 15:27:32.910150 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:32.910242 (+ 92us) log.cc:844] Serialized 35 byte log entry
0320 15:27:32.910310 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:32.910336 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:32.910343 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":236124,"reported_tablets":147,"spinlock_wait_cycles":160066944,"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":540,"apply.run_wall_time_us":538,"num_ops":3,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":853,"prepare.run_wall_time_us":853,"raft.queue_time_us":1285,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":284,"replication_time_us":4537}]]}
I20260320 15:27:34.267740 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_a42ece35a94a4c2b9c3bbfb885208c5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:34.268355 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a42ece35a94a4c2b9c3bbfb885208c5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:34.268432 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:34.255744 28890 rpcz_store.cc:269] 0320 15:27:30.675038 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:30.697566 (+ 22528us) service_pool.cc:225] Handling call
0320 15:27:30.731783 (+ 34217us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:33.050384 (+2318601us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.050396 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:34.221088 (+1170692us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.038215 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.038549 (+ 334us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.038562 (+ 13us) write_op.cc:435] Acquired schema lock
0320 15:27:33.038567 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.038781 (+ 214us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.038807 (+ 26us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.038817 (+ 10us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:33.038928 (+ 111us) tablet.cc:700] Row locks acquired
0320 15:27:33.038932 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:33.039004 (+ 72us) write_op.cc:270] Start()
0320 15:27:33.039062 (+ 58us) write_op.cc:276] Timestamp: P: 1774020453038951 usec, L: 0
0320 15:27:33.039069 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.039418 (+ 349us) log.cc:844] Serialized 527 byte log entry
0320 15:27:33.045378 (+ 5960us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.045572 (+ 194us) write_op.cc:301] APPLY: starting
0320 15:27:33.045599 (+ 27us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.045665 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.045730 (+ 65us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.045820 (+ 90us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.045876 (+ 56us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:33.045888 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:33.046013 (+ 125us) log.cc:844] Serialized 23 byte log entry
0320 15:27:33.046104 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.046137 (+ 33us) write_op.cc:454] Released schema lock
0320 15:27:33.046148 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":185870,"reported_tablets":144,"spinlock_wait_cycles":69393792,"child_traces":[["op",{"apply.queue_time_us":118,"apply.run_cpu_time_us":655,"apply.run_wall_time_us":655,"num_ops":1,"prepare.queue_time_us":3189,"prepare.run_cpu_time_us":1407,"prepare.run_wall_time_us":1407,"raft.queue_time_us":2217,"raft.run_cpu_time_us":455,"raft.run_wall_time_us":456,"replication_time_us":6255}]]}
I20260320 15:27:34.269440 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_eee4676e5c014be69777c4327813932e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:34.270041 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eee4676e5c014be69777c4327813932e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:34.273192 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41024969454c4959a48da8f41b8531c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:34.273346 28734 consensus_queue.cc:260] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.275972 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 9732504551294b3bb3369cdaacacb8d3 (DEFAULT_TABLE table=default.Table_215c6d41f475486a94794ebbd60809c1 [id=8972fc25652d4d2597b67852a355d132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.249228 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.276736 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.284184 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.284406 28736 tablet_bootstrap.cc:654] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.285454 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.286535 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.287946 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.289777 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.292775 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.294448 28734 ts_tablet_manager.cc:1434] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.029s user 0.001s sys 0.002s
I20260320 15:27:34.294984 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.295226 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.299052 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.299839 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.300127 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.300375 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.301272 29246 consensus_queue.cc:1048] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.301957 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (DEFAULT_TABLE table=default.Table_8f15c883f7c74571a7717c5dcdfb4f6d [id=e6144852663f4116bf276bf082e4c4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.302577 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 0a58d5350b8f407188b32786bfae2892 (DEFAULT_TABLE table=default.Table_c8c2dd201dc543b59a93b39cb64d5d83 [id=bced97ce4cc343ea806e2023813a498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.303663 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.304488 28734 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.305809 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.305861 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.307322 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 9732504551294b3bb3369cdaacacb8d3 (DEFAULT_TABLE table=default.Table_215c6d41f475486a94794ebbd60809c1 [id=8972fc25652d4d2597b67852a355d132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.307593 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.307956 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.308041 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:34.314821 28239 leader_election.cc:343] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.315097 28239 leader_election.cc:304] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.316987 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.317327 29677 raft_consensus.cc:2749] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.322439 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.323807 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (DEFAULT_TABLE table=default.Table_8f15c883f7c74571a7717c5dcdfb4f6d [id=e6144852663f4116bf276bf082e4c4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.325428 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.327219 28734 tablet_bootstrap.cc:654] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.327903 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.331983 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 0a58d5350b8f407188b32786bfae2892 (DEFAULT_TABLE table=default.Table_c8c2dd201dc543b59a93b39cb64d5d83 [id=bced97ce4cc343ea806e2023813a498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.332935 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.333554 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 9732504551294b3bb3369cdaacacb8d3 (DEFAULT_TABLE table=default.Table_215c6d41f475486a94794ebbd60809c1 [id=8972fc25652d4d2597b67852a355d132]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.334323 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 0a58d5350b8f407188b32786bfae2892 (DEFAULT_TABLE table=default.Table_c8c2dd201dc543b59a93b39cb64d5d83 [id=bced97ce4cc343ea806e2023813a498e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.335273 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (DEFAULT_TABLE table=default.Table_8f15c883f7c74571a7717c5dcdfb4f6d [id=e6144852663f4116bf276bf082e4c4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:34.344035 29751 tablet.cc:1723] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.344828 29640 ts_tablet_manager.cc:1929] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.348968 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 60) took 3444 ms. Trace:
I20260320 15:27:34.349282 28890 rpcz_store.cc:276] 0320 15:27:30.904743 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.269204 (+3364461us) service_pool.cc:225] Handling call
0320 15:27:34.329985 (+ 60781us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:34.330608 (+ 623us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:34.348519 (+ 17911us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:34.348634 (+ 115us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:34.348732 (+ 98us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.334993 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.335111 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.335118 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:34.335120 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:34.335198 (+ 78us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.335211 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.335217 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.335292 (+ 75us) tablet.cc:700] Row locks acquired
0320 15:27:34.335297 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:34.335339 (+ 42us) write_op.cc:270] Start()
0320 15:27:34.335372 (+ 33us) write_op.cc:276] Timestamp: P: 1774020454335308 usec, L: 0
0320 15:27:34.335374 (+ 2us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.335559 (+ 185us) log.cc:844] Serialized 433 byte log entry
0320 15:27:34.347256 (+ 11697us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.347905 (+ 649us) write_op.cc:301] APPLY: starting
0320 15:27:34.347923 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.347971 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.348007 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.348117 (+ 110us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.348150 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:34.348157 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:34.348258 (+ 101us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.348332 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.348353 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:34.348360 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.348437 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.348441 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":572,"apply.run_cpu_time_us":541,"apply.run_wall_time_us":541,"num_ops":2,"prepare.queue_time_us":3903,"prepare.run_cpu_time_us":647,"prepare.run_wall_time_us":646,"raft.queue_time_us":1948,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":9,"replication_time_us":11837}]]}
I20260320 15:27:34.353790 29640 log.cc:1199] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/89425da569794a8fa972817023bf853d
I20260320 15:27:34.356204 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 58) took 3511 ms. Trace:
I20260320 15:27:34.356727 29640 ts_tablet_manager.cc:1950] T 89425da569794a8fa972817023bf853d P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:34.356875 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9732504551294b3bb3369cdaacacb8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.356506 28900 rpcz_store.cc:276] 0320 15:27:30.845082 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.258320 (+3413238us) service_pool.cc:225] Handling call
0320 15:27:34.328717 (+ 70397us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:34.329398 (+ 681us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:34.355832 (+ 26434us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:34.355979 (+ 147us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:34.356118 (+ 139us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.335676 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.335776 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.335780 (+ 4us) write_op.cc:435] Acquired schema lock
0320 15:27:34.335784 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:34.335856 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.335865 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.335869 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.335924 (+ 55us) tablet.cc:700] Row locks acquired
0320 15:27:34.335926 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:34.335953 (+ 27us) write_op.cc:270] Start()
0320 15:27:34.335979 (+ 26us) write_op.cc:276] Timestamp: P: 1774020454335933 usec, L: 0
0320 15:27:34.335982 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.336118 (+ 136us) log.cc:844] Serialized 433 byte log entry
0320 15:27:34.347328 (+ 11210us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.348515 (+ 1187us) write_op.cc:301] APPLY: starting
0320 15:27:34.348529 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.348562 (+ 33us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.348583 (+ 21us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.348667 (+ 84us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.348692 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:34.348698 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:34.348756 (+ 58us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.348820 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.348837 (+ 17us) write_op.cc:454] Released schema lock
0320 15:27:34.348842 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1154,"apply.run_cpu_time_us":381,"apply.run_wall_time_us":381,"mutex_wait_us":3884,"num_ops":2,"prepare.queue_time_us":5751,"prepare.run_cpu_time_us":518,"prepare.run_wall_time_us":517,"raft.queue_time_us":1452,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":11329,"threads_started":1}]]}
I20260320 15:27:34.358537 28584 raft_consensus.cc:2468] T e13f1a65661949258f521dc847e674cd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:34.360164 28524 leader_election.cc:304] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.360793 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ab0f33ad6b4cdfbe36e707872bee95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.359205 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 59) took 3454 ms. Trace:
I20260320 15:27:34.364372 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a58d5350b8f407188b32786bfae2892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.361634 28905 rpcz_store.cc:276] 0320 15:27:30.904584 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.267405 (+3362821us) service_pool.cc:225] Handling call
0320 15:27:34.328469 (+ 61064us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:34.329256 (+ 787us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:34.358533 (+ 29277us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:34.358847 (+ 314us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:34.359058 (+ 211us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.333917 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.334100 (+ 183us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.334111 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:34.334115 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:34.334245 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.334266 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.334274 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.334385 (+ 111us) tablet.cc:700] Row locks acquired
0320 15:27:34.334389 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:34.334438 (+ 49us) write_op.cc:270] Start()
0320 15:27:34.334501 (+ 63us) write_op.cc:276] Timestamp: P: 1774020454334410 usec, L: 0
0320 15:27:34.334506 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.334796 (+ 290us) log.cc:844] Serialized 433 byte log entry
0320 15:27:34.338741 (+ 3945us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.338904 (+ 163us) write_op.cc:301] APPLY: starting
0320 15:27:34.338925 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.338986 (+ 61us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.339042 (+ 56us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.339221 (+ 179us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.339280 (+ 59us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:34.339292 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:34.339390 (+ 98us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.339462 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.339480 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:34.339486 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.339588 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.339593 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":672,"apply.run_wall_time_us":672,"num_ops":2,"prepare.queue_time_us":3964,"prepare.run_cpu_time_us":1064,"prepare.run_wall_time_us":1064,"raft.queue_time_us":362,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":274,"replication_time_us":4197}]]}
I20260320 15:27:34.367846 28905 master_service.cc:954] ReplaceTablet: received request to replace tablet f0a31d385b81456fb46ea52083874fcf from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:34.368121 28900 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_15707876735c420fb2d7fbc028f9b1d0" } new_table_name: "default.Table_43739f94773040b8a2e5986104bf32fc" modify_external_catalogs: true
W20260320 15:27:34.370728 28239 leader_election.cc:343] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.371035 28239 leader_election.cc:304] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.371651 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ab0f33ad6b4cdfbe36e707872bee95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.370806 29742 tablet.cc:1723] T d56de00174db4e2d8b1c7215038a04aa P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.387792 29754 tablet.cc:1723] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.386032 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet f0a31d385b81456fb46ea52083874fcf
I20260320 15:27:34.389595 28905 catalog_manager.cc:6741] ReplaceTablet: tablet f0a31d385b81456fb46ea52083874fcf deleted and replaced by tablet 60ad9b381aaa47f1afb63df266faee09
I20260320 15:27:34.389981 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:34.390058 28905 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 62) took 3463 ms. Trace:
I20260320 15:27:34.394413 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9732504551294b3bb3369cdaacacb8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.394647 29676 raft_consensus.cc:2749] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.394716 29706 raft_consensus.cc:2749] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.395925 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:34.390327 28905 rpcz_store.cc:276] 0320 15:27:30.926333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.367637 (+3441304us) service_pool.cc:225] Handling call
0320 15:27:34.389813 (+ 22176us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.372663 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.372772 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.372777 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:34.372779 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:34.372887 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.372899 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.372903 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.372969 (+ 66us) tablet.cc:700] Row locks acquired
0320 15:27:34.372971 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:34.373004 (+ 33us) write_op.cc:270] Start()
0320 15:27:34.373038 (+ 34us) write_op.cc:276] Timestamp: P: 1774020454372977 usec, L: 0
0320 15:27:34.373041 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.373202 (+ 161us) log.cc:844] Serialized 622 byte log entry
0320 15:27:34.380722 (+ 7520us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.381526 (+ 804us) write_op.cc:301] APPLY: starting
0320 15:27:34.381541 (+ 15us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.381583 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.381616 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.381748 (+ 132us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.381830 (+ 82us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:34.381836 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:34.381910 (+ 74us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.381960 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.381982 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:34.381990 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.386875 (+ 4885us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.386885 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":12032,"child_traces":[["op",{"apply.queue_time_us":764,"apply.run_cpu_time_us":536,"apply.run_wall_time_us":536,"mutex_wait_us":1485,"num_ops":2,"prepare.queue_time_us":2356,"prepare.run_cpu_time_us":5120,"prepare.run_wall_time_us":14186,"raft.queue_time_us":989,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":7662,"thread_start_us":12986,"threads_started":1}]]}
I20260320 15:27:34.403411 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_349b87ec6dc54907b74b3a5bf7861225"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:34.364360 29767 tablet.cc:1723] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.404812 28736 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:34.407107 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:34.408224 28733 tablet_bootstrap.cc:492] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.408560 28736 ts_tablet_manager.cc:1403] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.161s user 0.003s sys 0.010s
I20260320 15:27:34.408567 28733 ts_tablet_manager.cc:1403] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.280s user 0.011s sys 0.008s
I20260320 15:27:34.409405 28733 raft_consensus.cc:359] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.409492 28736 raft_consensus.cc:359] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.409607 28733 raft_consensus.cc:385] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.409679 28733 raft_consensus.cc:740] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.409925 28733 consensus_queue.cc:260] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.410516 28733 ts_tablet_manager.cc:1434] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:34.410858 28733 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.412277 29797 tablet.cc:1723] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260320 15:27:34.408456 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_349b87ec6dc54907b74b3a5bf7861225 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:34.412858 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a58d5350b8f407188b32786bfae2892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.419852 29617 raft_consensus.cc:493] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.420059 29617 raft_consensus.cc:515] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.420804 29617 leader_election.cc:290] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.421456 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.422780 28446 raft_consensus.cc:2468] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.423456 28524 leader_election.cc:304] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.423800 29617 raft_consensus.cc:2804] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:34.423913 29617 raft_consensus.cc:493] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:34.423995 29617 raft_consensus.cc:3060] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.424908 28733 tablet_bootstrap.cc:654] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.427577 28736 raft_consensus.cc:385] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.427743 28736 raft_consensus.cc:740] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.428045 28736 consensus_queue.cc:260] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.429379 28736 ts_tablet_manager.cc:1434] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.021s user 0.001s sys 0.001s
I20260320 15:27:34.432307 28736 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.433274 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 63) took 3363 ms. Trace:
I20260320 15:27:34.433522 28905 rpcz_store.cc:276] 0320 15:27:31.070066 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.403159 (+3333093us) service_pool.cc:225] Handling call
0320 15:27:34.419448 (+ 16289us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:34.420147 (+ 699us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:34.432867 (+ 12720us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.432876 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:34.432973 (+ 97us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:34.433101 (+ 128us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:34.433224 (+ 123us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.420804 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.420948 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.420955 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:34.420958 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:34.421072 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.421089 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.421093 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.421177 (+ 84us) tablet.cc:700] Row locks acquired
0320 15:27:34.421179 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:34.421222 (+ 43us) write_op.cc:270] Start()
0320 15:27:34.421262 (+ 40us) write_op.cc:276] Timestamp: P: 1774020454421188 usec, L: 0
0320 15:27:34.421266 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.421495 (+ 229us) log.cc:844] Serialized 433 byte log entry
0320 15:27:34.428436 (+ 6941us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.430652 (+ 2216us) write_op.cc:301] APPLY: starting
0320 15:27:34.430675 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.430728 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.430776 (+ 48us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.430927 (+ 151us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.430968 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:34.430976 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:34.431075 (+ 99us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.431152 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.431186 (+ 34us) write_op.cc:454] Released schema lock
0320 15:27:34.431194 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2099,"apply.run_cpu_time_us":619,"apply.run_wall_time_us":620,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":822,"prepare.run_wall_time_us":823,"raft.queue_time_us":1259,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":273,"replication_time_us":7130}]]}
I20260320 15:27:34.434372 28736 tablet_bootstrap.cc:654] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.435416 28905 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_3787678c407745418136143b1e78fd61" } modify_external_catalogs: true
I20260320 15:27:34.435750 28905 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_3787678c407745418136143b1e78fd61" } modify_external_catalogs: true
I20260320 15:27:34.437142 28583 raft_consensus.cc:2468] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:34.448645 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.448916 28301 raft_consensus.cc:2468] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.454720 29618 raft_consensus.cc:493] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.455044 29618 raft_consensus.cc:515] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.458083 29753 tablet.cc:1723] T 5b263f9a5df2477e8dc4cd52e5834991 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:34.459074 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.459321 28446 raft_consensus.cc:2468] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:34.459766 28733 tablet_bootstrap.cc:492] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.459930 28733 ts_tablet_manager.cc:1403] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.000s
I20260320 15:27:34.459996 28240 leader_election.cc:304] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.460448 29555 raft_consensus.cc:2804] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:34.460552 29555 raft_consensus.cc:493] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:34.460662 29555 raft_consensus.cc:3060] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.460930 28733 raft_consensus.cc:359] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.461120 28733 raft_consensus.cc:385] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.461223 28733 raft_consensus.cc:740] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.461544 28733 consensus_queue.cc:260] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.462476 28733 ts_tablet_manager.cc:1434] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:34.462808 28733 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:34.462827 28862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111) [suppressed 12 similar messages]
I20260320 15:27:34.465098 28733 tablet_bootstrap.cc:654] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.466903 29618 leader_election.cc:290] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.467434 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.474603 28734 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.474874 28734 ts_tablet_manager.cc:1403] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.180s user 0.001s sys 0.004s
I20260320 15:27:34.475958 28734 raft_consensus.cc:359] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.477946 28734 raft_consensus.cc:385] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.478086 28734 raft_consensus.cc:740] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.478339 28736 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:34.478526 28736 ts_tablet_manager.cc:1403] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.003s
I20260320 15:27:34.478544 28734 consensus_queue.cc:260] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.479346 28734 ts_tablet_manager.cc:1434] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:34.479295 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:34.483788 28734 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.485646 28734 tablet_bootstrap.cc:654] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.488047 28736 raft_consensus.cc:359] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.488327 28736 raft_consensus.cc:385] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.488341 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:34.488685 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:34.488904 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:34.489064 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:34.489504 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:34.489724 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:34.490248 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:34.488442 28736 raft_consensus.cc:740] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.491179 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:34.491581 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:34.491470 28736 consensus_queue.cc:260] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.492327 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:34.493307 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:34.493674 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:34.493875 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:34.494097 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:34.494424 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 1 got 0
I20260320 15:27:34.494616 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:34.495077 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:34.495453 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:34.496090 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:34.496711 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:34.497082 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:34.497292 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:34.497534 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:34.498162 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:34.499567 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:34.499876 29806 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.27.74.61:44739 every 8 connection(s)
I20260320 15:27:34.500114 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:34.500260 29423 rpc_server.cc:307] RPC server started. Bound to: 127.27.74.61:44739
I20260320 15:27:34.500468 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:34.500881 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:34.501180 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:34.501648 28734 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.501665 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:34.501847 28734 ts_tablet_manager.cc:1403] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.010s
I20260320 15:27:34.502008 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:34.502354 28889 catalog_manager.cc:5671] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } }
I20260320 15:27:34.502839 29423 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
I20260320 15:27:34.502882 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:34.502842 28734 raft_consensus.cc:359] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.503254 28734 raft_consensus.cc:385] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.503365 28734 raft_consensus.cc:740] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.503635 28736 ts_tablet_manager.cc:1434] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.020s user 0.002s sys 0.001s
I20260320 15:27:34.504040 28736 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.506435 28736 tablet_bootstrap.cc:654] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.507766 28734 consensus_queue.cc:260] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.509126 27944 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskc3SESu/build/asan/bin/kudu as pid 29423
I20260320 15:27:34.503551 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:34.512581 28733 tablet_bootstrap.cc:492] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.512779 28733 ts_tablet_manager.cc:1403] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260320 15:27:34.513051 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:34.513640 28733 raft_consensus.cc:359] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.513847 28733 raft_consensus.cc:385] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.513928 28733 raft_consensus.cc:740] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.514173 28733 consensus_queue.cc:260] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.517046 28734 ts_tablet_manager.cc:1434] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.015s user 0.000s sys 0.002s
I20260320 15:27:34.517971 28733 ts_tablet_manager.cc:1434] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:34.518345 28733 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.522301 28733 tablet_bootstrap.cc:654] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.522691 28734 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.527338 28734 tablet_bootstrap.cc:654] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.537842 29246 raft_consensus.cc:493] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.538019 29246 raft_consensus.cc:515] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.538902 29246 leader_election.cc:290] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.539911 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.540184 28302 raft_consensus.cc:2468] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:34.540617 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.540904 28583 raft_consensus.cc:2468] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:34.541285 29617 raft_consensus.cc:515] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.542037 29617 leader_election.cc:290] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.542455 28576 raft_consensus.cc:2393] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:34.549283 29555 raft_consensus.cc:515] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.550300 29555 leader_election.cc:290] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.550977 28383 leader_election.cc:304] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.556753 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:34.557156 28302 raft_consensus.cc:2393] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:34.561599 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9732504551294b3bb3369cdaacacb8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.561605 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:34.561904 28576 raft_consensus.cc:2393] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:34.562206 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a58d5350b8f407188b32786bfae2892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.562592 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ab0f33ad6b4cdfbe36e707872bee95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:34.568190 28733 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.568171 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:34.568409 28446 raft_consensus.cc:3060] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.568424 28733 ts_tablet_manager.cc:1403] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260320 15:27:34.574234 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6231fe30c5b34d0bb7a233abbffeb1e8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:34.574584 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:34.575201 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:34.575644 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:34.575856 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:34.576773 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:34.576994 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:34.577463 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:34.577940 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:34.579144 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:34.579604 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:34.579907 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:34.580161 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:34.580791 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:34.581270 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:34.581497 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:34.581709 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:34.582026 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:34.582298 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:34.582574 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:34.582952 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:34.583316 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:34.583508 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:34.583782 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:34.584019 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:34.584328 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:34.584561 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:34.584794 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:34.585044 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:34.585497 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:34.585736 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:34.586874 29626 raft_consensus.cc:2804] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:34.586972 29626 raft_consensus.cc:493] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:34.587049 29626 raft_consensus.cc:3060] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:34.602315 28382 consensus_peers.cc:597] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89425da569794a8fa972817023bf853d. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:34.603169 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 75) took 1028 ms. Trace:
I20260320 15:27:34.603586 28733 raft_consensus.cc:359] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.606011 28240 leader_election.cc:304] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.606523 29706 raft_consensus.cc:2749] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.606740 29618 raft_consensus.cc:493] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.603333 29640 rpcz_store.cc:276] 0320 15:27:33.550003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.553155 (+ 3152us) service_pool.cc:225] Handling call
0320 15:27:34.060992 (+507837us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:34.060999 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:34.156227 (+ 95228us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.344701 (+188474us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.578030 (+233329us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":458929}
I20260320 15:27:34.609228 28733 raft_consensus.cc:385] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.609306 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 89425da569794a8fa972817023bf853d (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:34.610961 29640 tablet_replica.cc:333] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:34.610756 28733 raft_consensus.cc:740] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.613726 29618 raft_consensus.cc:515] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.615301 28733 consensus_queue.cc:260] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.617532 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:34.618535 28240 leader_election.cc:343] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.616698 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.620118 29640 raft_consensus.cc:2243] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:34.620386 29618 leader_election.cc:290] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:34.620450 28239 leader_election.cc:343] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.620607 28239 leader_election.cc:304] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.621783 29618 raft_consensus.cc:2749] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.626581 28736 tablet_bootstrap.cc:492] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:34.626859 28736 ts_tablet_manager.cc:1403] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.123s user 0.011s sys 0.012s
I20260320 15:27:34.627601 29640 raft_consensus.cc:2272] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:34.627785 28736 raft_consensus.cc:359] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.628028 28736 raft_consensus.cc:385] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.632447 28736 raft_consensus.cc:740] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.635556 28736 consensus_queue.cc:260] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.637250 28733 ts_tablet_manager.cc:1434] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.067s user 0.004s sys 0.000s
I20260320 15:27:34.638309 28734 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.638554 28734 ts_tablet_manager.cc:1403] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.116s user 0.002s sys 0.006s
I20260320 15:27:34.639932 28734 raft_consensus.cc:359] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.640180 28734 raft_consensus.cc:385] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.640235 28734 raft_consensus.cc:740] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.640910 28736 ts_tablet_manager.cc:1434] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260320 15:27:34.641288 28734 consensus_queue.cc:260] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.641819 28734 ts_tablet_manager.cc:1434] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:34.642141 28734 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:34.643287 28736 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.644683 29640 ts_tablet_manager.cc:1916] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:34.648157 28736 tablet_bootstrap.cc:654] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.649216 28733 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.653961 28446 raft_consensus.cc:2468] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:34.659164 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:34.661465 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:34.662732 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:34.662956 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:34.663237 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:34.663466 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:34.663717 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:34.665494 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:34.666733 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:34.667286 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:34.667568 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:34.668385 28524 leader_election.cc:304] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:34.669207 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:34.669812 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:34.670378 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:34.670913 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:34.672061 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 1 got 0
I20260320 15:27:34.673282 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:34.673954 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:34.675626 29617 raft_consensus.cc:2804] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:34.675853 29617 raft_consensus.cc:697] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:34.678157 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:34.679364 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:34.682267 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:34.683136 28903 catalog_manager.cc:5671] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:34.684451 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:34.685237 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:34.676592 29617 consensus_queue.cc:237] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.685914 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:34.686450 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:34.687335 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:34.688113 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:34.688769 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:34.688822 28734 tablet_bootstrap.cc:654] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.689234 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:34.689430 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:34.689620 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:34.689896 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:34.690933 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:34.691828 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:34.692373 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:34.693091 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:34.693817 28903 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:34.701486 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:34.717535 28733 tablet_bootstrap.cc:654] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.727695 29676 raft_consensus.cc:493] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.727867 29676 raft_consensus.cc:515] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.728538 29676 leader_election.cc:290] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:34.729380 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bfb87d5d5741adb0ad2f2a88f4867f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.729604 28446 raft_consensus.cc:2468] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.730187 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bfb87d5d5741adb0ad2f2a88f4867f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.730407 28302 raft_consensus.cc:2468] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.731083 28524 leader_election.cc:304] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:34.731664 29676 raft_consensus.cc:2804] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:34.731751 29676 raft_consensus.cc:493] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:34.731813 29676 raft_consensus.cc:3060] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.741441 28733 tablet_bootstrap.cc:492] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.741664 28733 ts_tablet_manager.cc:1403] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.104s user 0.004s sys 0.000s
W20260320 15:27:34.746402 28870 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet fa54115d4df74bec85970057b5c3dd51 consensus state has no leader
I20260320 15:27:34.759222 29626 raft_consensus.cc:515] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.759965 29626 leader_election.cc:290] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.761082 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:34.761289 28302 raft_consensus.cc:3060] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.764760 28733 raft_consensus.cc:359] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.765038 28733 raft_consensus.cc:385] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.765111 28733 raft_consensus.cc:740] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.766011 28733 consensus_queue.cc:260] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.766584 28733 ts_tablet_manager.cc:1434] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.025s user 0.002s sys 0.002s
I20260320 15:27:34.766920 28733 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.770015 29673 raft_consensus.cc:493] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.770179 29673 raft_consensus.cc:515] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.770864 29673 leader_election.cc:290] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.771783 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.772047 28301 raft_consensus.cc:2393] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:34.772028 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca5e0c215f574b8aafec8d5e2da9a399" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:34.772215 28583 raft_consensus.cc:3060] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.772759 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7394e941c99e4358ad7a33db84b15348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.773332 28524 leader_election.cc:304] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:34.773706 29684 raft_consensus.cc:493] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.773820 29684 raft_consensus.cc:515] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.774374 29684 leader_election.cc:290] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:34.774499 29684 raft_consensus.cc:3060] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.775813 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:34.775985 28446 raft_consensus.cc:2468] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.776391 28524 leader_election.cc:304] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.776667 29599 raft_consensus.cc:2804] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:34.776743 29599 raft_consensus.cc:493] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:34.776803 29599 raft_consensus.cc:3060] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.777370 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.777562 28301 raft_consensus.cc:2468] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:34.779745 29673 consensus_queue.cc:1048] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.784521 29676 raft_consensus.cc:515] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.786268 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bfb87d5d5741adb0ad2f2a88f4867f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:34.786465 28297 raft_consensus.cc:3060] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.787505 28733 tablet_bootstrap.cc:654] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.789346 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4bfb87d5d5741adb0ad2f2a88f4867f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:34.789561 28443 raft_consensus.cc:3060] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.789357 29676 leader_election.cc:290] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:34.796841 28733 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.797137 28733 ts_tablet_manager.cc:1403] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.003s
I20260320 15:27:34.797938 28733 raft_consensus.cc:359] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.798130 28733 raft_consensus.cc:385] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.798185 28733 raft_consensus.cc:740] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.798427 28733 consensus_queue.cc:260] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.798921 28733 ts_tablet_manager.cc:1434] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:34.799292 28733 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.801079 28733 tablet_bootstrap.cc:654] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.806039 29246 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.806537 29246 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.810104 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.811484 29555 consensus_queue.cc:1048] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.812532 29246 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.813959 29246 raft_consensus.cc:493] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.814152 29246 raft_consensus.cc:515] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.815351 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:34.816074 28382 leader_election.cc:343] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.816776 29246 leader_election.cc:290] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.820523 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.820788 28301 raft_consensus.cc:2393] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:34.821069 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.821211 28383 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:34.821717 28383 leader_election.cc:343] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.821820 28383 leader_election.cc:304] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:34.822310 29626 raft_consensus.cc:3060] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.822638 29246 raft_consensus.cc:2749] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.828701 29807 sys_catalog.cc:263] Verifying existing consensus state
I20260320 15:27:34.834203 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:34.839386 29678 ts_tablet_manager.cc:1929] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.839578 29678 log.cc:1199] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:34.840054 29678 ts_tablet_manager.cc:1950] T a2775196c8e04dcc80c7e089b57579fd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:34.844187 28733 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.844347 28733 ts_tablet_manager.cc:1403] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.002s
I20260320 15:27:34.845085 28733 raft_consensus.cc:359] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.845276 28733 raft_consensus.cc:385] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.845323 28733 raft_consensus.cc:740] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.845598 29744 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:55030 (request call id 78) took 1210 ms. Trace:
I20260320 15:27:34.845728 28733 consensus_queue.cc:260] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.845691 29744 rpcz_store.cc:276] 0320 15:27:33.635212 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.635257 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:34.845575 (+1210318us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.754527 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0320 15:27:33.754662 (+ 135us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0320 15:27:33.754674 (+ 12us) alter_schema_op.cc:66] Acquired schema lock
0320 15:27:33.754688 (+ 14us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0320 15:27:33.754754 (+ 66us) alter_schema_op.cc:128] START. Timestamp: 1774020453754687
0320 15:27:33.754757 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.754974 (+ 217us) log.cc:844] Serialized 173 byte log entry
0320 15:27:34.093669 (+338695us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.253109 (+159440us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0320 15:27:34.253620 (+ 511us) log.cc:844] Serialized 5 byte log entry
0320 15:27:34.845164 (+591544us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.845470 (+ 306us) log.cc:844] Serialized 14 byte log entry
0320 15:27:34.845522 (+ 52us) alter_schema_op.cc:73] Released schema lock
0320 15:27:34.845528 (+ 6us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":159382,"prepare.queue_time_us":104541,"prepare.run_cpu_time_us":872,"prepare.run_wall_time_us":38933,"raft.queue_time_us":112,"raft.run_cpu_time_us":933,"raft.run_wall_time_us":2491,"replication_time_us":338868,"thread_start_us":166455,"threads_started":2,"wal-append.queue_time_us":701}]]}
I20260320 15:27:34.846218 28733 ts_tablet_manager.cc:1434] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:34.846688 28733 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:34.848145 28302 raft_consensus.cc:2468] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:34.849320 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 80) took 1020 ms. Trace:
I20260320 15:27:34.849406 29678 rpcz_store.cc:276] 0320 15:27:33.828630 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.828699 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:34.169305 (+340606us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:34.169315 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:34.365331 (+196016us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.839193 (+473862us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.849299 (+ 10106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":340384}
I20260320 15:27:34.849761 29678 tablet_replica.cc:333] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:34.849932 29678 raft_consensus.cc:2243] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:34.850224 29678 raft_consensus.cc:2272] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:34.850776 29678 ts_tablet_manager.cc:1916] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:34.853890 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet a2775196c8e04dcc80c7e089b57579fd (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]) successfully deleted
I20260320 15:27:34.862387 29706 consensus_queue.cc:1048] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.869256 28383 leader_election.cc:304] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.870383 29608 raft_consensus.cc:493] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.870535 29608 raft_consensus.cc:515] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.871162 29608 leader_election.cc:290] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:34.871238 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:34.871300 29608 raft_consensus.cc:2804] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:34.871379 29608 raft_consensus.cc:697] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:34.871676 29608 consensus_queue.cc:237] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:34.873679 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:34.864311 29599 raft_consensus.cc:515] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.877198 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906f207dee674a4288c8c60b5d3ff9f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.877489 28301 raft_consensus.cc:2393] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:34.886159 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:34.886391 28301 raft_consensus.cc:3060] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.886406 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e4a749f0744e68b502af97edeeb584" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:34.886619 28441 raft_consensus.cc:3060] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.886919 29599 leader_election.cc:290] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:34.887497 28443 raft_consensus.cc:2468] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:34.887969 28524 leader_election.cc:304] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:34.888734 28733 tablet_bootstrap.cc:654] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.893353 29608 consensus_queue.cc:1048] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.893857 29608 consensus_queue.cc:1048] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.894472 29599 raft_consensus.cc:2804] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:34.894757 29599 raft_consensus.cc:697] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:34.895016 29599 consensus_queue.cc:237] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.897665 28383 leader_election.cc:304] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:34.902586 29835 raft_consensus.cc:3060] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:34.912385 29608 raft_consensus.cc:493] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:34.912403 29744 tablet.cc:1723] T 4c6f8c20b5904eafb9a62d14bb155c82 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.912565 29608 raft_consensus.cc:515] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.913148 29608 leader_election.cc:290] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:34.913574 29751 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:34.914448 28733 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.914609 28733 ts_tablet_manager.cc:1403] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.068s user 0.000s sys 0.007s
I20260320 15:27:34.915621 28733 raft_consensus.cc:359] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.915865 28733 raft_consensus.cc:385] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.915926 28733 raft_consensus.cc:740] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.916172 28733 consensus_queue.cc:260] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.918063 29676 consensus_queue.cc:1048] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.930552 29754 tablet.cc:1723] T 4c6f8c20b5904eafb9a62d14bb155c82 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:34.930802 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:34.932403 29762 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:34.916831 28733 ts_tablet_manager.cc:1434] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:34.937405 28733 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:34.937417 28383 leader_election.cc:343] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:34.939224 28733 tablet_bootstrap.cc:654] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.939945 29643 ts_tablet_manager.cc:1929] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.940061 29643 log.cc:1199] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/edb7b53bb7444a39b471218c6039542f
I20260320 15:27:34.940431 29643 ts_tablet_manager.cc:1950] T edb7b53bb7444a39b471218c6039542f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:34.941421 29640 ts_tablet_manager.cc:1929] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:34.941630 29640 log.cc:1199] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:34.942095 29640 ts_tablet_manager.cc:1950] T adb65832eefe4bd4b08d95aa83c7bd7b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:34.945305 28583 raft_consensus.cc:2468] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:34.954806 28297 raft_consensus.cc:2468] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:34.956947 28736 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:34.957105 28736 ts_tablet_manager.cc:1403] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.316s user 0.057s sys 0.067s
I20260320 15:27:34.957967 28736 raft_consensus.cc:359] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.958127 28736 raft_consensus.cc:385] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.958171 28736 raft_consensus.cc:740] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.958415 28736 consensus_queue.cc:260] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.958950 28736 ts_tablet_manager.cc:1434] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:34.959228 28736 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:34.970916 29626 raft_consensus.cc:2749] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:34.976584 29617 consensus_queue.cc:1048] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.979148 28736 tablet_bootstrap.cc:654] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:34.971223 28734 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:34.981534 28734 ts_tablet_manager.cc:1403] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.338s user 0.003s sys 0.003s
I20260320 15:27:34.982659 28734 raft_consensus.cc:359] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.985682 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Bootstrap starting.
I20260320 15:27:34.987704 28734 raft_consensus.cc:385] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:34.988081 28734 raft_consensus.cc:740] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:34.988504 28734 consensus_queue.cc:260] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:34.990720 29599 consensus_queue.cc:1048] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:34.993613 28734 ts_tablet_manager.cc:1434] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260320 15:27:34.994716 28733 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:34.995643 28733 ts_tablet_manager.cc:1403] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.058s user 0.035s sys 0.017s
I20260320 15:27:35.008147 28734 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.011338 28734 tablet_bootstrap.cc:654] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.016955 28733 raft_consensus.cc:359] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.018288 28733 raft_consensus.cc:385] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.018954 28733 raft_consensus.cc:740] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.023164 29842 consensus_queue.cc:1048] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.023468 28736 tablet_bootstrap.cc:492] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:35.023788 28736 ts_tablet_manager.cc:1403] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.004s
I20260320 15:27:35.025985 28736 raft_consensus.cc:359] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.028672 28736 raft_consensus.cc:385] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.029265 28736 raft_consensus.cc:740] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.020985 28733 consensus_queue.cc:260] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.032789 29555 raft_consensus.cc:493] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.030985 28736 consensus_queue.cc:260] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.036098 29684 raft_consensus.cc:2749] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.036692 28734 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.036824 28734 ts_tablet_manager.cc:1403] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.002s
I20260320 15:27:35.037212 28441 raft_consensus.cc:2468] T f3e4a749f0744e68b502af97edeeb584 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:35.037684 28734 raft_consensus.cc:359] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.037838 28734 raft_consensus.cc:385] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.037882 28734 raft_consensus.cc:740] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.037427 28733 ts_tablet_manager.cc:1434] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.041s user 0.001s sys 0.003s
I20260320 15:27:35.038069 28734 consensus_queue.cc:260] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.038506 28734 ts_tablet_manager.cc:1434] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:35.038775 28734 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.038597 28736 ts_tablet_manager.cc:1434] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260320 15:27:35.039919 28733 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:35.040920 28736 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:35.041188 28734 tablet_bootstrap.cc:654] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.043264 28734 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.043387 28734 ts_tablet_manager.cc:1403] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:35.044175 28734 raft_consensus.cc:359] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.044346 28734 raft_consensus.cc:385] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.044389 28734 raft_consensus.cc:740] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.044384 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 77) took 1474 ms. Trace:
I20260320 15:27:35.044489 29640 rpcz_store.cc:276] 0320 15:27:33.570184 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.570586 (+ 402us) service_pool.cc:225] Handling call
0320 15:27:34.607960 (+1037374us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:34.607971 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:34.774923 (+166952us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.941336 (+166413us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.044360 (+103024us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1003055}
I20260320 15:27:35.044577 28734 consensus_queue.cc:260] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.044937 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 78) took 1441 ms. Trace:
I20260320 15:27:35.045012 29640 rpcz_store.cc:276] 0320 15:27:33.603564 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.603615 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:35.044789 (+1441174us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:35.044801 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.044921 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1437690}
I20260320 15:27:35.045135 28734 ts_tablet_manager.cc:1434] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:35.045262 29640 tablet_replica.cc:333] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:35.045405 28734 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.045459 29640 raft_consensus.cc:2243] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:35.045634 29640 raft_consensus.cc:2272] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.046255 29640 ts_tablet_manager.cc:1916] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.047014 28734 tablet_bootstrap.cc:654] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.049782 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet adb65832eefe4bd4b08d95aa83c7bd7b (table default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]) successfully deleted
W20260320 15:27:35.050144 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 71ec63fb080545de838803033f204fab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71ec63fb080545de838803033f204fab already in progress: creating tablet
I20260320 15:27:35.051048 28301 raft_consensus.cc:2468] T f3e4a749f0744e68b502af97edeeb584 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:35.052951 28733 tablet_bootstrap.cc:654] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.054163 28736 tablet_bootstrap.cc:654] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.055498 28524 leader_election.cc:304] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:35.056203 29684 consensus_queue.cc:1048] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.056756 29684 raft_consensus.cc:2804] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:35.056883 29684 raft_consensus.cc:697] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:35.057121 29684 consensus_queue.cc:237] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.032948 29555 raft_consensus.cc:515] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.062379 29555 leader_election.cc:290] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.063400 28734 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.063467 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:35.063601 28734 ts_tablet_manager.cc:1403] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.002s
W20260320 15:27:35.064395 28240 leader_election.cc:343] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.064504 28734 raft_consensus.cc:359] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.064699 28734 raft_consensus.cc:385] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.064769 28734 raft_consensus.cc:740] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.064997 28734 consensus_queue.cc:260] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.065640 28734 ts_tablet_manager.cc:1434] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:35.065984 28734 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.068382 28734 tablet_bootstrap.cc:654] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.075783 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:35.081692 28239 leader_election.cc:343] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.082455 28239 leader_election.cc:304] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:35.083149 29829 raft_consensus.cc:2749] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.087971 29841 consensus_queue.cc:1048] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.061s
I20260320 15:27:35.128005 29835 raft_consensus.cc:2749] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.132339 29678 ts_tablet_manager.cc:1929] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:35.132493 29678 log.cc:1199] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/edb7b53bb7444a39b471218c6039542f
I20260320 15:27:35.133873 29678 ts_tablet_manager.cc:1950] T edb7b53bb7444a39b471218c6039542f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:35.135675 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 83) took 1214 ms. Trace:
I20260320 15:27:35.135792 29643 rpcz_store.cc:276] 0320 15:27:33.921015 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.921225 (+ 210us) service_pool.cc:225] Handling call
0320 15:27:34.166675 (+245450us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:34.166690 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:34.748415 (+581725us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:34.939802 (+191387us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.135652 (+195850us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":245097}
I20260320 15:27:35.136503 29643 tablet_replica.cc:333] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:35.136735 29643 raft_consensus.cc:2243] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:35.136938 29643 raft_consensus.cc:2272] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.137298 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet edb7b53bb7444a39b471218c6039542f (table default.Table_1a8f81dd2a33405b819cad145fe64901 [id=7b5c27e0814148848d52b88c9dce9553]) successfully deleted
I20260320 15:27:35.137614 29643 ts_tablet_manager.cc:1916] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.150177 29617 consensus_queue.cc:1048] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.178994 29829 raft_consensus.cc:493] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.179652 29829 raft_consensus.cc:515] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:35.181670 28240 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:35.184690 29829 leader_election.cc:290] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:35.199219 29797 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:35.210250 29617 raft_consensus.cc:493] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.213754 29617 raft_consensus.cc:515] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.216212 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 82) took 1342 ms. Trace:
I20260320 15:27:35.216360 29678 rpcz_store.cc:276] 0320 15:27:33.874067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.879415 (+ 5348us) service_pool.cc:225] Handling call
0320 15:27:34.849713 (+970298us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:34.849724 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:34.939796 (+ 90072us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.132183 (+192387us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.216181 (+ 83998us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":928544}
I20260320 15:27:35.216732 29678 tablet_replica.cc:333] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:35.216960 29678 raft_consensus.cc:2243] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:35.217352 29678 raft_consensus.cc:2272] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.218032 29678 ts_tablet_manager.cc:1916] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.220031 29617 leader_election.cc:290] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.220667 29862 raft_consensus.cc:493] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:35.221944 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:35.222175 28441 raft_consensus.cc:2468] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:35.222954 28524 leader_election.cc:304] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:35.223703 29599 raft_consensus.cc:2804] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.223791 29599 raft_consensus.cc:493] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.223858 29599 raft_consensus.cc:3060] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.224910 29862 raft_consensus.cc:515] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:35.225884 29862 leader_election.cc:290] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739)
I20260320 15:27:35.240315 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 6 candidate_status { last_received { term: 5 index: 609 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:35.267431 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.272543 29640 ts_tablet_manager.cc:1929] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:35.272956 29640 log.cc:1199] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:35.273511 29640 ts_tablet_manager.cc:1950] T 66a181681cd84b7fb09c3742f4f2222f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:35.276451 28239 leader_election.cc:343] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:35.276834 28239 leader_election.cc:304] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:35.277207 29555 raft_consensus.cc:2749] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:35.283507 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 88439.7us
I20260320 15:27:35.284783 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet edb7b53bb7444a39b471218c6039542f (table default.Table_1a8f81dd2a33405b819cad145fe64901 [id=7b5c27e0814148848d52b88c9dce9553]) successfully deleted
I20260320 15:27:35.314617 29842 raft_consensus.cc:493] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.315745 29842 raft_consensus.cc:515] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.321482 29842 leader_election.cc:290] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:35.341462 28382 leader_election.cc:343] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.341742 28382 leader_election.cc:304] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:35.342092 29835 raft_consensus.cc:2749] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.347116 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:35.348816 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:35.370301 29835 consensus_queue.cc:1048] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.378405 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7bd0616f5604f8fbb8edd8e9c230f39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:35.378682 28301 raft_consensus.cc:2468] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:35.384621 28524 leader_election.cc:343] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.387573 28383 leader_election.cc:304] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:35.388056 29841 raft_consensus.cc:2804] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.388195 29841 raft_consensus.cc:493] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.388307 29841 raft_consensus.cc:3060] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.388939 29555 raft_consensus.cc:493] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.389091 29555 raft_consensus.cc:515] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.389771 29555 leader_election.cc:290] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:35.390841 28240 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:35.404999 29835 raft_consensus.cc:493] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.405210 29835 raft_consensus.cc:515] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.406121 29835 leader_election.cc:290] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:35.406456 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7bd0616f5604f8fbb8edd8e9c230f39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.406692 28575 raft_consensus.cc:2468] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:35.412204 29835 consensus_queue.cc:1048] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.417603 29618 raft_consensus.cc:493] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.417783 29618 raft_consensus.cc:515] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.417936 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:35.418078 28301 raft_consensus.cc:2468] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:35.418388 29618 leader_election.cc:290] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.419749 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:35.420324 28240 leader_election.cc:343] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.423381 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:35.424162 28382 leader_election.cc:343] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.426689 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:35.426959 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:35.427197 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
W20260320 15:27:35.428818 28239 leader_election.cc:343] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:35.428959 28239 leader_election.cc:304] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:35.429287 29829 raft_consensus.cc:2749] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.435895 28383 leader_election.cc:304] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:35.427455 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:35.436475 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:35.436893 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:35.437064 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:35.437356 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:35.437605 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:35.437887 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:35.438061 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 1 got 0
I20260320 15:27:35.438213 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]). Expected version 2 got 0
I20260320 15:27:35.438918 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:35.439374 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:35.439718 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:35.440135 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:35.440395 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.440539 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:35.440721 29842 raft_consensus.cc:2804] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.440861 29842 raft_consensus.cc:493] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.440917 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:35.441072 29842 raft_consensus.cc:3060] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.441238 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:35.441681 29684 raft_consensus.cc:493] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.441946 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
W20260320 15:27:35.442341 28239 leader_election.cc:343] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.442377 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:35.442502 28239 leader_election.cc:304] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:35.442764 29829 raft_consensus.cc:2749] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.442950 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]). Expected version 1 got 0
I20260320 15:27:35.443372 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:35.443635 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:35.443914 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:35.444145 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:35.444701 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:35.444919 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:35.445298 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:35.445600 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:35.445947 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:35.446319 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:35.446579 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:35.446813 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:35.447279 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:35.450420 29684 raft_consensus.cc:515] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.451647 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:35.452239 28524 leader_election.cc:343] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.454664 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:35.454918 28446 raft_consensus.cc:2468] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:35.455614 28524 leader_election.cc:304] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:35.455966 29617 raft_consensus.cc:2804] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.456048 29617 raft_consensus.cc:493] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.456121 29617 raft_consensus.cc:3060] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.458810 29684 leader_election.cc:290] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.463189 28734 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.463384 28734 ts_tablet_manager.cc:1403] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.397s user 0.051s sys 0.165s
I20260320 15:27:35.464388 28734 raft_consensus.cc:359] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.464615 28734 raft_consensus.cc:385] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.464687 28734 raft_consensus.cc:740] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.464936 28734 consensus_queue.cc:260] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.465890 28734 ts_tablet_manager.cc:1434] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:35.466252 28734 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.465833 29868 tablet.cc:1723] T 4c6f8c20b5904eafb9a62d14bb155c82 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:35.479466 28734 tablet_bootstrap.cc:654] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.526080 28422 tablet_service.cc:1511] Processing CreateTablet for tablet c456e9eff7904004830bcda3efa77dbf (DEFAULT_TABLE table=default.Table_4e32eac691b748468ec7ad07338699b7 [id=0ccdd080968542e197d15da3c810e4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.526196 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 7840b529206d482283cc550d2c5e7606 (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.527009 28274 tablet_service.cc:1511] Processing CreateTablet for tablet c456e9eff7904004830bcda3efa77dbf (DEFAULT_TABLE table=default.Table_4e32eac691b748468ec7ad07338699b7 [id=0ccdd080968542e197d15da3c810e4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.527817 28551 tablet_service.cc:1511] Processing CreateTablet for tablet c456e9eff7904004830bcda3efa77dbf (DEFAULT_TABLE table=default.Table_4e32eac691b748468ec7ad07338699b7 [id=0ccdd080968542e197d15da3c810e4c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.528559 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 7840b529206d482283cc550d2c5e7606 (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.530820 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9705754d6c4ab8a2f674dc359df83f (DEFAULT_TABLE table=default.Table_14baef3186e14a0ab4705c1cdaff1717 [id=ae5198db1f3547d99914fee671b0aabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.532282 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9705754d6c4ab8a2f674dc359df83f (DEFAULT_TABLE table=default.Table_14baef3186e14a0ab4705c1cdaff1717 [id=ae5198db1f3547d99914fee671b0aabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.532676 29829 raft_consensus.cc:493] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.532823 29829 raft_consensus.cc:515] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.533474 29829 leader_election.cc:290] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.534029 28266 tablet_service.cc:1511] Processing CreateTablet for tablet 7840b529206d482283cc550d2c5e7606 (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.534891 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:35.535138 28446 raft_consensus.cc:2468] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
W20260320 15:27:35.535348 29865 log.cc:927] Time spent T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Append to log took a long time: real 0.108s user 0.000s sys 0.000s
I20260320 15:27:35.535944 28240 leader_election.cc:304] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:35.536442 28734 tablet_bootstrap.cc:492] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.536623 28734 ts_tablet_manager.cc:1403] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.070s user 0.001s sys 0.003s
I20260320 15:27:35.536791 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.537340 29829 raft_consensus.cc:2804] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.537369 28734 raft_consensus.cc:359] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.537443 29829 raft_consensus.cc:493] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.537533 29829 raft_consensus.cc:3060] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.537550 28734 raft_consensus.cc:385] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.537598 28734 raft_consensus.cc:740] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.537782 28734 consensus_queue.cc:260] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.538267 28734 ts_tablet_manager.cc:1434] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:35.538564 28734 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:35.538594 28239 leader_election.cc:343] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.540704 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 7e9705754d6c4ab8a2f674dc359df83f (DEFAULT_TABLE table=default.Table_14baef3186e14a0ab4705c1cdaff1717 [id=ae5198db1f3547d99914fee671b0aabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.541419 28734 tablet_bootstrap.cc:654] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:35.543944 29858 log.cc:927] Time spent T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Append to log took a long time: real 0.188s user 0.000s sys 0.000s
I20260320 15:27:35.561749 28734 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.562258 28734 ts_tablet_manager.cc:1403] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.005s
I20260320 15:27:35.564208 28734 raft_consensus.cc:359] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.564664 28734 raft_consensus.cc:385] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.564810 28734 raft_consensus.cc:740] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.565410 28734 consensus_queue.cc:260] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.567188 28734 ts_tablet_manager.cc:1434] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:35.568156 28734 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.571652 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:35.572003 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:35.572252 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:35.572494 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:35.572669 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:35.572985 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d56de00174db4e2d8b1c7215038a04aa (table default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c]). Expected version 2 got 0
I20260320 15:27:35.573130 28734 tablet_bootstrap.cc:654] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.573172 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7f94140a5e8b4a06890bca86456d64a6 (table default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]). Expected version 1 got 0
I20260320 15:27:35.573385 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:35.573797 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:35.574247 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:35.575276 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8f6f683638874e90b0d3c8969c702a5c (table default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe]). Expected version 4 got 0
I20260320 15:27:35.575515 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_923e214b36464c938d8b5be225e0a94b [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 1 got 0
I20260320 15:27:35.576253 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:35.576468 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:35.577033 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4b4040a0214a46b580b2ac2de59243ce (table default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76]). Expected version 2 got 0
I20260320 15:27:35.577298 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:35.577535 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:35.578245 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5b263f9a5df2477e8dc4cd52e5834991 (table default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e]). Expected version 1 got 0
I20260320 15:27:35.578861 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:35.579088 28734 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.579458 28734 ts_tablet_manager.cc:1403] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260320 15:27:35.579648 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:35.579865 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:35.580065 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:35.580380 28890 catalog_manager.cc:5671] T e13f1a65661949258f521dc847e674cd P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:35.580904 28734 raft_consensus.cc:359] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.581105 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:35.581212 28734 raft_consensus.cc:385] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.581306 28734 raft_consensus.cc:740] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.581722 28734 consensus_queue.cc:260] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.582046 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:35.582384 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]). Expected version 2 got 0
I20260320 15:27:35.582778 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:35.582788 28734 ts_tablet_manager.cc:1434] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:35.582949 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:35.583194 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:35.583321 28734 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.583842 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:35.584029 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:35.584362 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:35.584666 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:35.584892 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:35.585227 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:35.585462 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:35.585691 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:35.586078 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:35.586483 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 8d92e17cea044541968356ae993be1e7
W20260320 15:27:35.586694 28902 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4190329df1b644c392ff8cbace9aae7a consensus state has no leader
I20260320 15:27:35.586795 28734 tablet_bootstrap.cc:654] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.586925 28902 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 57) took 4742 ms. Trace:
I20260320 15:27:35.587178 28892 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 53) took 4743 ms. Trace:
I20260320 15:27:35.590245 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:35.590271 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.587328 28892 rpcz_store.cc:276] 0320 15:27:30.843902 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.716213 (+2872311us) service_pool.cc:225] Handling call
0320 15:27:33.728630 (+ 12417us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:33.728815 (+ 185us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:33.728912 (+ 97us) catalog_manager.cc:2824] Locking tablets
0320 15:27:33.729074 (+ 162us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:33.816647 (+ 87573us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.816660 (+ 13us) write_op.cc:454] Released schema lock
0320 15:27:33.816808 (+ 148us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:33.846534 (+ 29726us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:35.586397 (+1739863us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:35.587098 (+ 701us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.790141 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.790380 (+ 239us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.790392 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:33.790397 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:33.790585 (+ 188us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.790609 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.790617 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:33.790750 (+ 133us) tablet.cc:700] Row locks acquired
0320 15:27:33.790752 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:33.790800 (+ 48us) write_op.cc:270] Start()
0320 15:27:33.790841 (+ 41us) write_op.cc:276] Timestamp: P: 1774020453790776 usec, L: 0
0320 15:27:33.790844 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.791140 (+ 296us) log.cc:844] Serialized 756 byte log entry
0320 15:27:33.800959 (+ 9819us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.801159 (+ 200us) write_op.cc:301] APPLY: starting
0320 15:27:33.801187 (+ 28us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.801265 (+ 78us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.801306 (+ 41us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.801413 (+ 107us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.801467 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:33.801479 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:33.801595 (+ 116us) log.cc:844] Serialized 29 byte log entry
0320 15:27:33.801682 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.801717 (+ 35us) write_op.cc:454] Released schema lock
0320 15:27:33.801729 (+ 12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":130,"apply.run_cpu_time_us":645,"apply.run_wall_time_us":645,"mutex_wait_us":31020,"num_ops":2,"prepare.queue_time_us":31276,"prepare.run_cpu_time_us":1159,"prepare.run_wall_time_us":1158,"raft.queue_time_us":561,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":343,"replication_time_us":10058}]]}
I20260320 15:27:35.591015 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:35.593360 28900 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f6240d441c624b238f7c13eb3c5fa9fc consensus state has no leader
I20260320 15:27:35.593611 28900 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 61) took 4685 ms. Trace:
I20260320 15:27:35.593719 28900 rpcz_store.cc:276] 0320 15:27:30.908590 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.358018 (+3449428us) service_pool.cc:225] Handling call
0320 15:27:34.367982 (+ 9964us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:34.368616 (+ 634us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:34.369057 (+ 441us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:34.369446 (+ 389us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:34.369615 (+ 169us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:34.385355 (+ 15740us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:35.593540 (+1208185us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.371861 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.371979 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.371987 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:34.371990 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:34.372099 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.372118 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.372126 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:34.372202 (+ 76us) tablet.cc:700] Row locks acquired
0320 15:27:34.372204 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:34.372246 (+ 42us) write_op.cc:270] Start()
0320 15:27:34.372288 (+ 42us) write_op.cc:276] Timestamp: P: 1774020454372217 usec, L: 0
0320 15:27:34.372292 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.372504 (+ 212us) log.cc:844] Serialized 405 byte log entry
0320 15:27:34.380629 (+ 8125us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.380902 (+ 273us) write_op.cc:301] APPLY: starting
0320 15:27:34.380925 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.380973 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.381016 (+ 43us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.381076 (+ 60us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.381110 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:34.381117 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:34.381215 (+ 98us) log.cc:844] Serialized 23 byte log entry
0320 15:27:34.381290 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.381313 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:34.381320 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.381417 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.381423 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":20542208,"child_traces":[["op",{"apply.queue_time_us":214,"apply.run_cpu_time_us":498,"apply.run_wall_time_us":498,"num_ops":1,"prepare.queue_time_us":1538,"prepare.run_cpu_time_us":769,"prepare.run_wall_time_us":769,"raft.queue_time_us":1968,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":275,"replication_time_us":8299}]]}
I20260320 15:27:35.587026 28902 rpcz_store.cc:276] 0320 15:27:30.844399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.113504 (+3269105us) service_pool.cc:225] Handling call
0320 15:27:34.260148 (+146644us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:34.260509 (+ 361us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:34.260753 (+ 244us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:34.260835 (+ 82us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:34.271850 (+ 11015us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:35.586851 (+1315001us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.261584 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.261712 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.261721 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:34.261723 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:34.261838 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.261857 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.261862 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:34.261930 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:34.261933 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:34.261972 (+ 39us) write_op.cc:270] Start()
0320 15:27:34.262026 (+ 54us) write_op.cc:276] Timestamp: P: 1774020454261908 usec, L: 0
0320 15:27:34.262030 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.262236 (+ 206us) log.cc:844] Serialized 405 byte log entry
0320 15:27:34.267361 (+ 5125us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.267510 (+ 149us) write_op.cc:301] APPLY: starting
0320 15:27:34.267563 (+ 53us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.267616 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.267674 (+ 58us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.267747 (+ 73us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.267786 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:34.267793 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:34.267890 (+ 97us) log.cc:844] Serialized 23 byte log entry
0320 15:27:34.267963 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.267987 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:34.267994 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.268074 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.268078 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":552,"apply.run_wall_time_us":551,"num_ops":1,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":786,"raft.queue_time_us":841,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":292,"replication_time_us":5291,"spinlock_wait_cycles":6528}]]}
I20260320 15:27:35.608814 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:35.609786 28734 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.609942 28734 ts_tablet_manager.cc:1403] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.006s
I20260320 15:27:35.610859 28734 raft_consensus.cc:359] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.611057 28734 raft_consensus.cc:385] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.611119 28734 raft_consensus.cc:740] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.611344 28734 consensus_queue.cc:260] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.611887 28734 ts_tablet_manager.cc:1434] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:35.612208 28734 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.614367 28734 tablet_bootstrap.cc:654] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.617184 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:35.618058 28734 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.618381 28734 ts_tablet_manager.cc:1403] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.005s
I20260320 15:27:35.619834 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:35.620518 28734 raft_consensus.cc:359] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.620905 28734 raft_consensus.cc:385] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.621031 28734 raft_consensus.cc:740] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.621462 28734 consensus_queue.cc:260] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.622593 28734 ts_tablet_manager.cc:1434] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260320 15:27:35.623302 28734 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.621758 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:35.635612 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:35.645027 29807 log.cc:826] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Log is configured to *not* fsync() on all Append() calls
I20260320 15:27:35.645671 29829 raft_consensus.cc:515] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.646090 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9705754d6c4ab8a2f674dc359df83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.647387 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9705754d6c4ab8a2f674dc359df83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.647429 29643 ts_tablet_manager.cc:1929] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:35.648269 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c456e9eff7904004830bcda3efa77dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.648808 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7840b529206d482283cc550d2c5e7606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.649051 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:35.649261 28446 raft_consensus.cc:3060] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.649762 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:35.650450 28239 leader_election.cc:343] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.652518 29829 leader_election.cc:290] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.657373 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c456e9eff7904004830bcda3efa77dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.658046 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7840b529206d482283cc550d2c5e7606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.660066 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c456e9eff7904004830bcda3efa77dbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.660638 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7840b529206d482283cc550d2c5e7606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.661083 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e9705754d6c4ab8a2f674dc359df83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.667655 29643 log.cc:1199] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:35.668372 28905 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 64) took 4309 ms. Trace:
I20260320 15:27:35.668500 28905 rpcz_store.cc:276] 0320 15:27:31.358420 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.435208 (+3076788us) service_pool.cc:225] Handling call
0320 15:27:34.435916 (+ 708us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:34.436070 (+ 154us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:34.436136 (+ 66us) catalog_manager.cc:2824] Locking tablets
0320 15:27:34.526945 (+ 90809us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:34.539603 (+ 12658us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:34.539723 (+ 120us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:35.607868 (+1068145us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:35.668305 (+ 60437us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.531558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.531739 (+ 181us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.531749 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:34.531754 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:34.531937 (+ 183us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.531959 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.531968 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:34.532122 (+ 154us) tablet.cc:700] Row locks acquired
0320 15:27:34.532126 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:34.532177 (+ 51us) write_op.cc:270] Start()
0320 15:27:34.532224 (+ 47us) write_op.cc:276] Timestamp: P: 1774020454532131 usec, L: 0
0320 15:27:34.532230 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.532557 (+ 327us) log.cc:844] Serialized 759 byte log entry
0320 15:27:34.537775 (+ 5218us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.537967 (+ 192us) write_op.cc:301] APPLY: starting
0320 15:27:34.537992 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.538058 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.538110 (+ 52us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.538220 (+ 110us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.538274 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:34.538286 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:34.538396 (+ 110us) log.cc:844] Serialized 29 byte log entry
0320 15:27:34.538478 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.538521 (+ 43us) write_op.cc:454] Released schema lock
0320 15:27:34.538531 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.538627 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.538633 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":122,"apply.run_cpu_time_us":645,"apply.run_wall_time_us":643,"num_ops":2,"prepare.queue_time_us":80,"prepare.run_cpu_time_us":1171,"prepare.run_wall_time_us":1172,"raft.queue_time_us":613,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":392,"replication_time_us":5496}]]}
I20260320 15:27:35.669373 28905 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_bc31ab0980a04953909f5c8d56ae39ee" } modify_external_catalogs: true
I20260320 15:27:35.669559 28905 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_bc31ab0980a04953909f5c8d56ae39ee" } modify_external_catalogs: true
I20260320 15:27:35.671180 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.676302 29840 raft_consensus.cc:493] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:35.676373 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 113) took 3884 ms (client timeout 1000 ms). Trace:
I20260320 15:27:35.676520 29840 raft_consensus.cc:515] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.677232 29840 leader_election.cc:290] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:35.676502 28904 rpcz_store.cc:269] 0320 15:27:31.792266 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:32.792017 (+999751us) service_pool.cc:225] Handling call
0320 15:27:35.676298 (+2884281us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:33.436419 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:33.436566 (+ 147us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:33.436573 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:33.436575 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:33.436721 (+ 146us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:33.436738 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:33.436745 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:33.442119 (+ 5374us) tablet.cc:700] Row locks acquired
0320 15:27:33.442124 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:33.442184 (+ 60us) write_op.cc:270] Start()
0320 15:27:33.442251 (+ 67us) write_op.cc:276] Timestamp: P: 1774020453442134 usec, L: 0
0320 15:27:33.442258 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:33.442553 (+ 295us) log.cc:844] Serialized 527 byte log entry
0320 15:27:33.448291 (+ 5738us) op_driver.cc:464] REPLICATION: finished
0320 15:27:33.448460 (+ 169us) write_op.cc:301] APPLY: starting
0320 15:27:33.448480 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:33.448535 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:33.448579 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:33.448661 (+ 82us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:33.448714 (+ 53us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:33.448725 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:33.448845 (+ 120us) log.cc:844] Serialized 23 byte log entry
0320 15:27:33.448933 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.448965 (+ 32us) write_op.cc:454] Released schema lock
0320 15:27:33.448974 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:33.449073 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:33.449079 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":77315,"reported_tablets":154,"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":598,"apply.run_wall_time_us":597,"num_ops":1,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":1079,"prepare.run_wall_time_us":6279,"raft.queue_time_us":1520,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":357,"replication_time_us":5987}]]}
I20260320 15:27:35.677125 29684 consensus_queue.cc:1048] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.677556 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet ffe278739bbc4ddab848af9c25241380 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:35.678002 28904 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 66) took 4209 ms. Trace:
I20260320 15:27:35.678164 28904 rpcz_store.cc:276] 0320 15:27:31.468997 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.677442 (+4208445us) service_pool.cc:225] Handling call
0320 15:27:35.677955 (+ 513us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:35.678577 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.678771 28575 raft_consensus.cc:2468] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:35.678968 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet 6771bd6a3f0e47d99a0191eae25cb2d2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:35.679325 28382 leader_election.cc:304] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:35.680029 29835 raft_consensus.cc:2804] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:35.680150 29835 raft_consensus.cc:493] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:35.680274 29835 raft_consensus.cc:3060] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.682914 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:35.683030 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:35.657159 28734 tablet_bootstrap.cc:654] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.682935 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:35.684124 28302 raft_consensus.cc:2468] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:35.709030 28734 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.709497 28734 ts_tablet_manager.cc:1403] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.086s user 0.000s sys 0.008s
I20260320 15:27:35.711681 28734 raft_consensus.cc:359] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.711995 28423 tablet_service.cc:1511] Processing CreateTablet for tablet c217b54730ea46c88ec01a03755b83cc (DEFAULT_TABLE table=default.Table_349b87ec6dc54907b74b3a5bf7861225 [id=0b4ddf8a4a1b4a19a42da0444fa72479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.712391 28734 raft_consensus.cc:385] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.712602 28734 raft_consensus.cc:740] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.713515 28734 consensus_queue.cc:260] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.714371 29840 raft_consensus.cc:493] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.714573 29840 raft_consensus.cc:515] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.715514 29840 leader_election.cc:290] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:35.716532 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.717376 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:35.718112 28382 leader_election.cc:343] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:35.718655 28383 leader_election.cc:343] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.718812 28383 leader_election.cc:304] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:35.719349 29840 raft_consensus.cc:2749] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:35.720237 28734 ts_tablet_manager.cc:1434] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.002s sys 0.001s
I20260320 15:27:35.720628 28734 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.722676 28734 tablet_bootstrap.cc:654] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.725091 28734 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.725281 28734 ts_tablet_manager.cc:1403] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260320 15:27:35.726264 28734 raft_consensus.cc:359] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.726547 28734 raft_consensus.cc:385] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:35.726994 29866 log.cc:927] Time spent T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.247s user 0.000s sys 0.000s
W20260320 15:27:35.727725 29772 log.cc:927] Time spent T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260320 15:27:35.728735 28736 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:35.728850 28736 ts_tablet_manager.cc:1403] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.688s user 0.015s sys 0.041s
I20260320 15:27:35.729583 28736 raft_consensus.cc:359] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.729744 28736 raft_consensus.cc:385] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.729796 28736 raft_consensus.cc:740] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.730036 28736 consensus_queue.cc:260] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.730567 28736 ts_tablet_manager.cc:1434] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:35.730886 28736 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:35.731002 28733 tablet_bootstrap.cc:492] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:35.731127 28733 ts_tablet_manager.cc:1403] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.692s user 0.015s sys 0.042s
I20260320 15:27:35.731799 29643 ts_tablet_manager.cc:1950] T b395977eb0444bcf8563cd8774fab40f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:35.731972 28733 raft_consensus.cc:359] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.732129 28733 raft_consensus.cc:385] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.732179 28733 raft_consensus.cc:740] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.732393 28733 consensus_queue.cc:260] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.732679 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 84) took 1792 ms. Trace:
I20260320 15:27:35.732878 28733 ts_tablet_manager.cc:1434] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:35.732796 29643 rpcz_store.cc:276] 0320 15:27:33.939826 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.940042 (+ 216us) service_pool.cc:225] Handling call
0320 15:27:35.136427 (+1196385us) ts_tablet_manager.cc:1151] Deleting tablet b395977eb0444bcf8563cd8774fab40f
0320 15:27:35.136447 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.231808 (+ 95361us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.647282 (+415474us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.732605 (+ 85323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1193165}
I20260320 15:27:35.733146 29643 tablet_replica.cc:333] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:35.733161 28733 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:35.733355 29643 raft_consensus.cc:2243] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:35.733556 29643 raft_consensus.cc:2272] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.734098 29643 ts_tablet_manager.cc:1916] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.735466 28423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c217b54730ea46c88ec01a03755b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.735575 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b395977eb0444bcf8563cd8774fab40f (table default.Table_bce978bcf8414d928d5f77eaf4e90cbe [id=95ea602214894df19afd570e9cf0258d]) successfully deleted
I20260320 15:27:35.740150 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 6771bd6a3f0e47d99a0191eae25cb2d2
I20260320 15:27:35.740605 28904 catalog_manager.cc:6741] ReplaceTablet: tablet 6771bd6a3f0e47d99a0191eae25cb2d2 deleted and replaced by tablet aaefdda7483e4c21b09b7c868395c6f1
I20260320 15:27:35.740756 28904 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 67) took 4242 ms. Trace:
I20260320 15:27:35.740844 28904 rpcz_store.cc:276] 0320 15:27:31.498768 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.678899 (+4180131us) service_pool.cc:225] Handling call
0320 15:27:35.740693 (+ 61794us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.712671 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.712847 (+ 176us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.712857 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:35.712862 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:35.713008 (+ 146us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.713028 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.713035 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:35.713144 (+ 109us) tablet.cc:700] Row locks acquired
0320 15:27:35.713148 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:35.713195 (+ 47us) write_op.cc:270] Start()
0320 15:27:35.713252 (+ 57us) write_op.cc:276] Timestamp: P: 1774020455713137 usec, L: 0
0320 15:27:35.713257 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.713560 (+ 303us) log.cc:844] Serialized 622 byte log entry
0320 15:27:35.719120 (+ 5560us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.725035 (+ 5915us) write_op.cc:301] APPLY: starting
0320 15:27:35.725062 (+ 27us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.725115 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.734525 (+ 9410us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.734660 (+ 135us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.734729 (+ 69us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:35.734738 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:35.734838 (+ 100us) log.cc:844] Serialized 29 byte log entry
0320 15:27:35.734927 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.734951 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:35.734960 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.735072 (+ 112us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.735076 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5808,"apply.run_cpu_time_us":679,"apply.run_wall_time_us":10016,"num_ops":2,"prepare.queue_time_us":1283,"prepare.run_cpu_time_us":1016,"prepare.run_wall_time_us":1016,"raft.queue_time_us":1662,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":408,"replication_time_us":5828}]]}
I20260320 15:27:35.741616 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet 16e31e11048b494ebe1fd4d89488b0cf from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:35.742332 28906 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4bd1799035a3431092fbdfad3c42d22b consensus state has no leader
W20260320 15:27:35.742815 28906 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet db16d86664794305bde714a7ff4d5caf consensus state has no leader
I20260320 15:27:35.743042 28274 tablet_service.cc:1511] Processing CreateTablet for tablet c217b54730ea46c88ec01a03755b83cc (DEFAULT_TABLE table=default.Table_349b87ec6dc54907b74b3a5bf7861225 [id=0b4ddf8a4a1b4a19a42da0444fa72479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.743922 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 956494c03c36462da65323aeccf65ccf (DEFAULT_TABLE table=default.Table_41024969454c4959a48da8f41b8531c2 [id=787f04a1d30b44b183b956fe2fc16af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.744441 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 11033b84b4234b6995c528c6d869cf5f (DEFAULT_TABLE table=default.Table_eee4676e5c014be69777c4327813932e [id=65429b1eb86c4fefb21b7abeb59c29d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.744480 28552 tablet_service.cc:1511] Processing CreateTablet for tablet c217b54730ea46c88ec01a03755b83cc (DEFAULT_TABLE table=default.Table_349b87ec6dc54907b74b3a5bf7861225 [id=0b4ddf8a4a1b4a19a42da0444fa72479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.745388 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 60ad9b381aaa47f1afb63df266faee09 (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.746009 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 87cb35cefee841d0881b08c456df5a97 (DEFAULT_TABLE table=default.Table_a42ece35a94a4c2b9c3bbfb885208c5b [id=febb68e38844460c857089e7afdf4276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.746354 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.747834 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 11033b84b4234b6995c528c6d869cf5f (DEFAULT_TABLE table=default.Table_eee4676e5c014be69777c4327813932e [id=65429b1eb86c4fefb21b7abeb59c29d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.748289 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11033b84b4234b6995c528c6d869cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.752100 28420 tablet_service.cc:1511] Processing CreateTablet for tablet 60ad9b381aaa47f1afb63df266faee09 (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.752636 28420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ad9b381aaa47f1afb63df266faee09. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:35.753403 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f6240d441c624b238f7c13eb3c5fa9fc consensus state has no leader
I20260320 15:27:35.755990 28426 tablet_service.cc:1511] Processing CreateTablet for tablet 87cb35cefee841d0881b08c456df5a97 (DEFAULT_TABLE table=default.Table_a42ece35a94a4c2b9c3bbfb885208c5b [id=febb68e38844460c857089e7afdf4276]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.756457 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87cb35cefee841d0881b08c456df5a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.756569 29678 ts_tablet_manager.cc:1929] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:35.756700 29678 log.cc:1199] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b34048a722c3485490637965c60cc5dc
I20260320 15:27:35.757110 29678 ts_tablet_manager.cc:1950] T b34048a722c3485490637965c60cc5dc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:35.761332 28410 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:35.766784 29684 consensus_queue.cc:1048] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:35.766947 29599 consensus_meta.cc:287] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Time spent flushing consensus metadata: real 0.543s user 0.003s sys 0.000s
I20260320 15:27:35.767027 29599 raft_consensus.cc:515] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.767648 29599 leader_election.cc:290] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:35.769029 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 956494c03c36462da65323aeccf65ccf (DEFAULT_TABLE table=default.Table_41024969454c4959a48da8f41b8531c2 [id=787f04a1d30b44b183b956fe2fc16af6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:35.771103 28906 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 8eb3d5e19cda48fda124b6238902c03a consensus state has no leader
I20260320 15:27:35.771477 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.772531 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.772913 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.773754 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet b8f3ee3c611d41b483bd35f868042d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.773924 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.774331 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c217b54730ea46c88ec01a03755b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.774667 28567 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.774665 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956494c03c36462da65323aeccf65ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.774837 28553 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:35.774991 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87cb35cefee841d0881b08c456df5a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.775919 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ad9b381aaa47f1afb63df266faee09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.778887 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11033b84b4234b6995c528c6d869cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.782490 28736 tablet_bootstrap.cc:654] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.783517 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 79) took 2178 ms. Trace:
I20260320 15:27:35.783643 29640 rpcz_store.cc:276] 0320 15:27:33.605032 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.605153 (+ 121us) service_pool.cc:225] Handling call
0320 15:27:35.045222 (+1440069us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:35.045232 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.152376 (+107144us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.272378 (+120002us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.783488 (+511110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1432282}
I20260320 15:27:35.784201 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 80) took 2177 ms. Trace:
I20260320 15:27:35.784299 29640 rpcz_store.cc:276] 0320 15:27:33.606716 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.606795 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:35.784040 (+2177245us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:35.784051 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.784182 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2157831}
I20260320 15:27:35.784603 29640 tablet_replica.cc:333] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:35.784811 29640 raft_consensus.cc:2243] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:35.785009 29640 raft_consensus.cc:2272] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.785691 29640 ts_tablet_manager.cc:1916] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.788851 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:35.789039 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:35.789203 28382 consensus_peers.cc:597] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:35.789491 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 956494c03c36462da65323aeccf65ccf (DEFAULT_TABLE table=default.Table_41024969454c4959a48da8f41b8531c2 [id=787f04a1d30b44b183b956fe2fc16af6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.789621 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 11033b84b4234b6995c528c6d869cf5f (DEFAULT_TABLE table=default.Table_eee4676e5c014be69777c4327813932e [id=65429b1eb86c4fefb21b7abeb59c29d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.790185 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 60ad9b381aaa47f1afb63df266faee09 (DEFAULT_TABLE table=default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:35.790525 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 87cb35cefee841d0881b08c456df5a97 (DEFAULT_TABLE table=default.Table_a42ece35a94a4c2b9c3bbfb885208c5b [id=febb68e38844460c857089e7afdf4276]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:35.793989 29880 log.cc:927] Time spent T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260320 15:27:35.797608 28733 tablet_bootstrap.cc:654] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.800583 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c217b54730ea46c88ec01a03755b83cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:35.804664 28524 leader_election.cc:343] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.805826 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 66a181681cd84b7fb09c3742f4f2222f (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:35.806838 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956494c03c36462da65323aeccf65ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.807094 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87cb35cefee841d0881b08c456df5a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.807329 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:35.807838 28892 catalog_manager.cc:5671] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:35.808295 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:35.808568 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:35.808763 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:35.809033 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:35.809407 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:35.809679 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:35.726656 28734 raft_consensus.cc:740] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.809981 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
W20260320 15:27:35.809729 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 92) took 1966 ms (client timeout 1000 ms). Trace:
I20260320 15:27:35.810488 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60ad9b381aaa47f1afb63df266faee09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.813613 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11033b84b4234b6995c528c6d869cf5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.810308 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:35.821429 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:35.821687 29599 consensus_queue.cc:1048] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.821902 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:35.822382 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:35.822540 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:35.822862 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:35.826352 28736 tablet_bootstrap.cc:492] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:35.826514 28736 ts_tablet_manager.cc:1403] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.002s
I20260320 15:27:35.827317 28736 raft_consensus.cc:359] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.827486 28736 raft_consensus.cc:385] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260320 15:27:35.810285 28906 rpcz_store.cc:269] 0320 15:27:33.842562 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.855150 (+ 12588us) service_pool.cc:225] Handling call
0320 15:27:35.809446 (+1954296us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":29186,"reported_tablets":198}
I20260320 15:27:35.829342 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_ec3f886f78ed4f3092a8d4e6cf282475" } new_table_name: "default.Table_313ec95c683c4f99adb04f3968de7963" modify_external_catalogs: true
I20260320 15:27:35.832774 28734 consensus_queue.cc:260] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:35.835937 29599 consensus_queue.cc:1048] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.842417 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 84) took 1932 ms. Trace:
I20260320 15:27:35.823465 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:35.844143 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956494c03c36462da65323aeccf65ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:35.844903 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:35.846450 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:35.848076 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:35.843101 28736 raft_consensus.cc:740] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.848270 29841 raft_consensus.cc:515] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.848397 29887 raft_consensus.cc:493] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:35.848603 29887 raft_consensus.cc:515] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.848680 28736 consensus_queue.cc:260] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.848975 29841 leader_election.cc:290] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:35.850029 28736 ts_tablet_manager.cc:1434] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.023s user 0.000s sys 0.003s
I20260320 15:27:35.850184 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:35.850529 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:35.851022 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:35.851468 28892 catalog_manager.cc:5671] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:35.851913 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:35.852139 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:35.852421 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:35.852717 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:35.852919 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:35.853250 28736 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:35.853271 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:35.853487 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:35.853704 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:35.853996 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:35.854348 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:35.854367 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26818e620ff649d18b671e4f4c7c7e74" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:35.854609 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:35.854645 28734 ts_tablet_manager.cc:1434] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.129s user 0.002s sys 0.002s
I20260320 15:27:35.855183 28736 tablet_bootstrap.cc:654] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.855657 28734 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:35.859680 28382 leader_election.cc:343] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:35.860190 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7bd0616f5604f8fbb8edd8e9c230f39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:35.860405 28575 raft_consensus.cc:3060] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:35.850239 29887 leader_election.cc:290] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:35.843148 29678 rpcz_store.cc:276] 0320 15:27:33.909596 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.909751 (+ 155us) service_pool.cc:225] Handling call
0320 15:27:35.216669 (+1306918us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:35.216679 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.646007 (+429328us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.756431 (+110424us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.842384 (+ 85953us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1292595}
I20260320 15:27:35.863714 28734 tablet_bootstrap.cc:654] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.865123 29643 ts_tablet_manager.cc:1929] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:35.866912 29643 log.cc:1199] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:35.867379 29643 ts_tablet_manager.cc:1950] T c752fb99ef494bda9d144d0e180e3100 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:35.868582 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:35.868800 28441 raft_consensus.cc:3060] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:35.869207 28109 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.27.74.61:44739 timed out after 0.640s (SENT) [suppressed 2 similar messages]
W20260320 15:27:35.890086 29807 log.cc:927] Time spent T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Append to log took a long time: real 0.055s user 0.000s sys 0.001s
I20260320 15:27:35.901872 29840 consensus_queue.cc:1048] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:35.904779 28904 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 68) took 2581 ms. Trace:
I20260320 15:27:35.905006 28904 rpcz_store.cc:276] 0320 15:27:33.323688 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.741488 (+2417800us) service_pool.cc:225] Handling call
0320 15:27:35.787075 (+ 45587us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:35.904704 (+117629us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":103881088}
I20260320 15:27:35.905648 28904 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_12977445848d4334ab141542db268e90" } modify_external_catalogs: true
I20260320 15:27:35.905930 28904 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_12977445848d4334ab141542db268e90" } modify_external_catalogs: true
I20260320 15:27:35.907459 28734 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.907874 28734 ts_tablet_manager.cc:1403] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.000s
I20260320 15:27:35.909119 28734 raft_consensus.cc:359] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.909472 28734 raft_consensus.cc:385] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.909615 28734 raft_consensus.cc:740] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
W20260320 15:27:35.921736 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 3ccbf4f32636463f8e1e005a629b92cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ccbf4f32636463f8e1e005a629b92cc already in progress: creating tablet
I20260320 15:27:35.923269 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:35.925246 28734 consensus_queue.cc:260] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.941231 28734 ts_tablet_manager.cc:1434] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260320 15:27:35.947360 28734 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:35.951328 28734 tablet_bootstrap.cc:654] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:35.964265 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b34048a722c3485490637965c60cc5dc (table default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]) successfully deleted
I20260320 15:27:35.966008 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 85) took 2055 ms. Trace:
I20260320 15:27:35.966152 29678 rpcz_store.cc:276] 0320 15:27:33.910101 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.910147 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:35.963132 (+2052985us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:35.963151 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.965970 (+ 2819us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2038731}
I20260320 15:27:35.966559 29678 tablet_replica.cc:333] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
W20260320 15:27:35.967339 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
I20260320 15:27:35.967711 29678 raft_consensus.cc:2243] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:35.968530 28734 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:35.969578 29678 raft_consensus.cc:2272] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:35.970283 29678 ts_tablet_manager.cc:1916] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:35.970597 28734 ts_tablet_manager.cc:1403] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20260320 15:27:35.988655 28734 raft_consensus.cc:359] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.989405 28734 raft_consensus.cc:385] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:35.989634 28734 raft_consensus.cc:740] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:35.990260 28734 consensus_queue.cc:260] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:35.994050 28734 ts_tablet_manager.cc:1434] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20260320 15:27:35.995972 28734 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:36.002942 28109 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 133754us
W20260320 15:27:36.007118 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 231947us
I20260320 15:27:36.008641 28733 tablet_bootstrap.cc:492] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.008823 28733 ts_tablet_manager.cc:1403] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.276s user 0.007s sys 0.009s
I20260320 15:27:36.010325 28733 raft_consensus.cc:359] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.010725 28733 raft_consensus.cc:385] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.010799 28733 raft_consensus.cc:740] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.011090 28733 consensus_queue.cc:260] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.011860 28733 ts_tablet_manager.cc:1434] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:36.012228 28733 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.013754 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7bd0616f5604f8fbb8edd8e9c230f39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.014067 28301 raft_consensus.cc:3060] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.019802 28734 tablet_bootstrap.cc:654] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.022430 28733 tablet_bootstrap.cc:654] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.031841 28734 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.032050 28734 ts_tablet_manager.cc:1403] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.001s
I20260320 15:27:36.033100 28734 raft_consensus.cc:359] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.033350 28734 raft_consensus.cc:385] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.033437 28734 raft_consensus.cc:740] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.036844 28734 consensus_queue.cc:260] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.037671 28734 ts_tablet_manager.cc:1434] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:36.038084 28734 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.040684 28734 tablet_bootstrap.cc:654] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:36.041129 29617 consensus_meta.cc:287] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Time spent flushing consensus metadata: real 0.585s user 0.002s sys 0.001s
I20260320 15:27:36.041265 29617 raft_consensus.cc:515] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.044251 29617 leader_election.cc:290] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.047876 28733 tablet_bootstrap.cc:492] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.048060 28733 ts_tablet_manager.cc:1403] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.001s
I20260320 15:27:36.049110 28733 raft_consensus.cc:359] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.049324 28733 raft_consensus.cc:385] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.049387 28733 raft_consensus.cc:740] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.049329 28446 raft_consensus.cc:2468] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:36.050127 28733 consensus_queue.cc:260] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.050988 28733 ts_tablet_manager.cc:1434] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260320 15:27:36.051025 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:36.051222 28444 raft_consensus.cc:3060] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.051340 28733 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.051683 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed29146bff74e86a366d3c6329176f4" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.051863 28302 raft_consensus.cc:3060] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.052544 29676 consensus_queue.cc:1048] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:36.053498 28733 tablet_bootstrap.cc:654] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.055462 28240 leader_election.cc:304] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.056217 29555 raft_consensus.cc:2804] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.056344 29555 raft_consensus.cc:697] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:36.056562 29555 consensus_queue.cc:237] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.062197 29835 raft_consensus.cc:515] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.063135 29835 leader_election.cc:290] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:36.063957 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.064147 28583 raft_consensus.cc:3060] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.067301 29835 raft_consensus.cc:493] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.067452 29835 raft_consensus.cc:515] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.069674 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:36.070282 28382 leader_election.cc:343] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.071554 29835 leader_election.cc:290] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.073882 28733 tablet_bootstrap.cc:492] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.074061 28733 ts_tablet_manager.cc:1403] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20260320 15:27:36.075198 28733 raft_consensus.cc:359] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.075439 28733 raft_consensus.cc:385] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.075503 28733 raft_consensus.cc:740] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.075806 28733 consensus_queue.cc:260] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.076431 28733 ts_tablet_manager.cc:1434] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:36.076793 28733 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.078950 28733 tablet_bootstrap.cc:654] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.085242 28734 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.085440 28734 ts_tablet_manager.cc:1403] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.047s user 0.000s sys 0.004s
I20260320 15:27:36.086395 28734 raft_consensus.cc:359] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.086619 28734 raft_consensus.cc:385] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.086681 28734 raft_consensus.cc:740] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.086903 28734 consensus_queue.cc:260] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.087452 28734 ts_tablet_manager.cc:1434] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:36.087841 28734 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.088449 29617 consensus_queue.cc:1048] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:36.090183 28734 tablet_bootstrap.cc:654] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.090183 29868 tablet.cc:1723] T ca5e0c215f574b8aafec8d5e2da9a399 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:36.101822 28302 raft_consensus.cc:2468] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.102618 28524 leader_election.cc:304] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:36.103081 29617 raft_consensus.cc:2804] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.103237 29617 raft_consensus.cc:697] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:36.103502 29617 consensus_queue.cc:237] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.107864 28583 raft_consensus.cc:2468] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.112735 28734 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.112943 28734 ts_tablet_manager.cc:1403] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.005s
I20260320 15:27:36.114161 28734 raft_consensus.cc:359] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.114408 28734 raft_consensus.cc:385] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.114480 28734 raft_consensus.cc:740] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.114725 28734 consensus_queue.cc:260] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.115376 28734 ts_tablet_manager.cc:1434] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260320 15:27:36.119901 29842 consensus_meta.cc:287] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Time spent flushing consensus metadata: real 0.678s user 0.003s sys 0.000s
I20260320 15:27:36.120363 29842 raft_consensus.cc:515] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.122778 28382 leader_election.cc:304] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.123571 29887 raft_consensus.cc:493] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.123725 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 85) took 2173 ms. Trace:
I20260320 15:27:36.123814 29886 raft_consensus.cc:2804] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.123804 29643 rpcz_store.cc:276] 0320 15:27:33.950380 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.950680 (+ 300us) service_pool.cc:225] Handling call
0320 15:27:35.733058 (+1782378us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:35.733068 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.843665 (+110597us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:35.864151 (+ 20486us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.123709 (+259558us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1782037}
I20260320 15:27:36.124148 29643 tablet_replica.cc:333] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:36.124341 29886 raft_consensus.cc:697] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.124846 29842 leader_election.cc:290] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.130002 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet c752fb99ef494bda9d144d0e180e3100 (table default.Table_be682052d66448438f9d7e7fc47a7ced [id=f330238fc7c246bcb43c472c5f931720]) successfully deleted
I20260320 15:27:36.130627 28733 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.130828 28733 ts_tablet_manager.cc:1403] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.000s
I20260320 15:27:36.132330 28733 raft_consensus.cc:359] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.132615 28733 raft_consensus.cc:385] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.132683 28733 raft_consensus.cc:740] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.133054 28733 consensus_queue.cc:260] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.133580 29678 ts_tablet_manager.cc:1929] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.133752 29678 log.cc:1199] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:36.134130 28733 ts_tablet_manager.cc:1434] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:36.134244 29678 ts_tablet_manager.cc:1950] T c752fb99ef494bda9d144d0e180e3100 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:36.134523 28733 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.134836 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.135078 28583 raft_consensus.cc:3060] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.137007 28733 tablet_bootstrap.cc:654] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.138236 28734 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.138303 29640 ts_tablet_manager.cc:1929] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.138579 29640 log.cc:1199] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b1bc86d3d1ff4b3aa8463eb21f619917
I20260320 15:27:36.139123 29640 ts_tablet_manager.cc:1950] T b1bc86d3d1ff4b3aa8463eb21f619917 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.143127 29886 consensus_queue.cc:237] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:36.144145 28239 consensus_peers.cc:597] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:36.144855 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:36.145994 29887 raft_consensus.cc:515] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.151608 29887 leader_election.cc:290] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.152086 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:36.152855 28382 leader_election.cc:343] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.152876 29643 raft_consensus.cc:2243] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.153121 29643 raft_consensus.cc:2272] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.159034 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26818e620ff649d18b671e4f4c7c7e74" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.159308 28297 raft_consensus.cc:2468] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:36.161031 29643 ts_tablet_manager.cc:1916] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.162783 28734 tablet_bootstrap.cc:654] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.165931 29842 raft_consensus.cc:493] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.166391 29842 raft_consensus.cc:515] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.167322 29842 leader_election.cc:290] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:36.167711 28524 consensus_peers.cc:597] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c752fb99ef494bda9d144d0e180e3100. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:36.168251 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:36.171835 28382 leader_election.cc:343] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.182154 29842 consensus_queue.cc:1048] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:36.185307 28734 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.185487 28734 ts_tablet_manager.cc:1403] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.006s
I20260320 15:27:36.186687 29909 tablet.cc:1723] T ca5e0c215f574b8aafec8d5e2da9a399 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:36.187389 28383 leader_election.cc:304] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.188366 28734 raft_consensus.cc:359] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.188565 28734 raft_consensus.cc:385] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.188618 28734 raft_consensus.cc:740] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.188817 28734 consensus_queue.cc:260] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.192821 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7feb8477ebf841a19977a434f365f537" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.193056 28297 raft_consensus.cc:3060] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.194068 29676 raft_consensus.cc:493] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:36.194216 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.194237 29676 raft_consensus.cc:515] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.194973 29676 leader_election.cc:290] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.195139 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea95b7a1b30e42cebcbfcd551c58f679" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.195317 28302 raft_consensus.cc:3060] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.189709 29842 raft_consensus.cc:2804] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.195706 29842 raft_consensus.cc:493] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.195992 29842 raft_consensus.cc:3060] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.196122 28734 ts_tablet_manager.cc:1434] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
W20260320 15:27:36.197212 28524 leader_election.cc:343] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:36.197389 28734 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:36.197423 28524 leader_election.cc:343] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:36.197537 28524 leader_election.cc:304] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.197916 29676 raft_consensus.cc:2749] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:36.199061 28736 tablet_bootstrap.cc:492] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.199290 28734 tablet_bootstrap.cc:654] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.199344 28736 ts_tablet_manager.cc:1403] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.346s user 0.029s sys 0.043s
I20260320 15:27:36.200919 28736 raft_consensus.cc:359] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.201228 28736 raft_consensus.cc:385] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.201313 28736 raft_consensus.cc:740] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.202769 28733 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.202917 28733 ts_tablet_manager.cc:1403] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.003s
I20260320 15:27:36.203727 28733 raft_consensus.cc:359] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.203894 28733 raft_consensus.cc:385] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.203941 28733 raft_consensus.cc:740] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.204138 28733 consensus_queue.cc:260] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.204624 28733 ts_tablet_manager.cc:1434] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:36.204720 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 83) took 2296 ms. Trace:
I20260320 15:27:36.204939 28733 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.204829 29678 rpcz_store.cc:276] 0320 15:27:33.908186 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.908504 (+ 318us) service_pool.cc:225] Handling call
0320 15:27:35.966489 (+2057985us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:35.966501 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.070751 (+104250us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.133234 (+ 62483us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.204696 (+ 71462us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2041134}
I20260320 15:27:36.205206 29678 tablet_replica.cc:333] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:36.205411 29678 raft_consensus.cc:2243] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.211174 28575 raft_consensus.cc:2468] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.212371 28733 tablet_bootstrap.cc:654] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.212551 28441 raft_consensus.cc:2468] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.201802 28736 consensus_queue.cc:260] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.217644 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.220530 28736 ts_tablet_manager.cc:1434] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.021s user 0.002s sys 0.001s
I20260320 15:27:36.222468 28382 leader_election.cc:304] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.222944 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet c752fb99ef494bda9d144d0e180e3100 (table default.Table_be682052d66448438f9d7e7fc47a7ced [id=f330238fc7c246bcb43c472c5f931720]) successfully deleted
I20260320 15:27:36.224015 29840 raft_consensus.cc:2804] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.224222 28736 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.224936 29840 raft_consensus.cc:697] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.225237 29840 consensus_queue.cc:237] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.226619 28524 leader_election.cc:304] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:36.227816 29617 raft_consensus.cc:2804] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.228030 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.228451 29617 raft_consensus.cc:697] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:36.228766 29617 consensus_queue.cc:237] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.229679 29835 raft_consensus.cc:493] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.229806 29835 raft_consensus.cc:515] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.230448 29835 leader_election.cc:290] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.231359 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "796c907919df47bf9369e45a1248f96b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.231766 28733 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.231933 28733 ts_tablet_manager.cc:1403] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.004s
W20260320 15:27:36.231945 28382 leader_election.cc:343] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.232841 28301 raft_consensus.cc:2468] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.233001 28733 raft_consensus.cc:359] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.233147 28733 raft_consensus.cc:385] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.233206 28733 raft_consensus.cc:740] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.233405 28733 consensus_queue.cc:260] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.234716 28733 ts_tablet_manager.cc:1434] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:36.237911 28734 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.238090 28734 ts_tablet_manager.cc:1403] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.041s user 0.000s sys 0.004s
I20260320 15:27:36.238226 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:36.238936 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:36.239082 28734 raft_consensus.cc:359] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.238909 28733 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.239271 28734 raft_consensus.cc:385] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.239326 28734 raft_consensus.cc:740] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.239583 28734 consensus_queue.cc:260] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.240130 29676 raft_consensus.cc:493] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.240295 28734 ts_tablet_manager.cc:1434] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:36.240267 29676 raft_consensus.cc:515] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.240643 28734 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.240975 29676 leader_election.cc:290] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.245993 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.246243 28443 raft_consensus.cc:2468] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.246624 28524 leader_election.cc:304] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.246918 29676 raft_consensus.cc:2804] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.246989 29676 raft_consensus.cc:493] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.247053 29676 raft_consensus.cc:3060] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.248183 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.248401 28301 raft_consensus.cc:2468] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.245358 28736 tablet_bootstrap.cc:654] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.251108 28734 tablet_bootstrap.cc:654] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.253973 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:36.254211 28383 leader_election.cc:343] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.254432 28383 leader_election.cc:304] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.254849 29891 raft_consensus.cc:2749] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:36.255643 28383 leader_election.cc:343] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.255863 28383 leader_election.cc:304] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.256992 28733 tablet_bootstrap.cc:654] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.257751 29891 raft_consensus.cc:493] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.257921 29891 raft_consensus.cc:515] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.258687 29891 leader_election.cc:290] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:36.262423 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.261447 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:36.264182 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "796c907919df47bf9369e45a1248f96b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:36.266640 28382 leader_election.cc:343] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.267849 29886 raft_consensus.cc:2749] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.268328 28736 tablet_bootstrap.cc:492] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.268570 28736 ts_tablet_manager.cc:1403] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.000s
I20260320 15:27:36.269671 28736 raft_consensus.cc:359] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.269865 28736 raft_consensus.cc:385] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.269918 28736 raft_consensus.cc:740] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.270262 28736 consensus_queue.cc:260] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.271256 29555 raft_consensus.cc:493] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.271409 29555 raft_consensus.cc:515] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.269960 28902 catalog_manager.cc:5671] T f3e4a749f0744e68b502af97edeeb584 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
W20260320 15:27:36.272205 28383 leader_election.cc:343] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.272794 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.272941 28383 leader_election.cc:304] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.273823 29555 leader_election.cc:290] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.274451 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce8595157679444b959b569dbb7d8df0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.274680 28443 raft_consensus.cc:2393] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:36.275171 28240 leader_election.cc:304] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.276049 29555 raft_consensus.cc:3060] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.276974 29887 raft_consensus.cc:2749] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.278441 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:36.282155 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:36.282790 28734 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.282605 28736 ts_tablet_manager.cc:1434] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260320 15:27:36.282936 28734 ts_tablet_manager.cc:1403] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.009s
I20260320 15:27:36.284049 28736 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.284016 28734 raft_consensus.cc:359] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.284232 28734 raft_consensus.cc:385] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.284305 28734 raft_consensus.cc:740] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.284521 28734 consensus_queue.cc:260] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.286440 28733 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.286623 28733 ts_tablet_manager.cc:1403] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.004s
I20260320 15:27:36.287467 28733 raft_consensus.cc:359] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.287876 28733 raft_consensus.cc:385] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.288050 28733 raft_consensus.cc:740] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.288435 28733 consensus_queue.cc:260] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.289034 28734 ts_tablet_manager.cc:1434] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260320 15:27:36.288582 28383 leader_election.cc:343] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.290261 29926 tablet.cc:1723] T ca5e0c215f574b8aafec8d5e2da9a399 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260320 15:27:36.289629 28383 leader_election.cc:304] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.289361 28734 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.291366 29887 raft_consensus.cc:2749] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.292419 28733 ts_tablet_manager.cc:1434] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:36.293908 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:36.295146 28733 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.298144 28734 tablet_bootstrap.cc:654] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.302110 28736 tablet_bootstrap.cc:654] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.302628 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:36.303838 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.303771 29643 ts_tablet_manager.cc:1929] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.307782 29643 log.cc:1199] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b34048a722c3485490637965c60cc5dc
I20260320 15:27:36.307767 28583 raft_consensus.cc:2468] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.308223 28302 raft_consensus.cc:2468] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.308133 29643 ts_tablet_manager.cc:1950] T b34048a722c3485490637965c60cc5dc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:36.308117 28383 leader_election.cc:343] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.308674 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 86) took 2336 ms. Trace:
I20260320 15:27:36.308918 29643 rpcz_store.cc:276] 0320 15:27:33.971951 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.972023 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:36.124098 (+2152075us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:36.124109 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.207042 (+ 82933us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.303614 (+ 96572us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.308659 (+ 5045us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28514,"tablet-delete.queue_time_us":2151722}
I20260320 15:27:36.309351 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 87) took 2330 ms. Trace:
I20260320 15:27:36.309413 29643 rpcz_store.cc:276] 0320 15:27:33.979333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.981726 (+ 2393us) service_pool.cc:225] Handling call
0320 15:27:36.309235 (+2327509us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:36.309243 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.309337 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2327232}
I20260320 15:27:36.309636 29643 tablet_replica.cc:333] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:36.309801 29643 raft_consensus.cc:2243] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.309939 29643 raft_consensus.cc:2272] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.310256 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b34048a722c3485490637965c60cc5dc (table default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]) successfully deleted
I20260320 15:27:36.310460 29643 ts_tablet_manager.cc:1916] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:36.310585 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
I20260320 15:27:36.309068 28383 leader_election.cc:304] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.311470 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 81) took 2695 ms. Trace:
I20260320 15:27:36.311584 29640 rpcz_store.cc:276] 0320 15:27:33.615925 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.616240 (+ 315us) service_pool.cc:225] Handling call
0320 15:27:35.784560 (+2168320us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:35.784569 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:35.950937 (+166368us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.138106 (+187169us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.311447 (+173341us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2114418}
I20260320 15:27:36.312079 28382 leader_election.cc:304] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.312336 29887 raft_consensus.cc:2804] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.312489 29887 raft_consensus.cc:697] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.312717 29887 consensus_queue.cc:237] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.312865 28734 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.313055 28734 ts_tablet_manager.cc:1403] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.003s
I20260320 15:27:36.313540 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b1bc86d3d1ff4b3aa8463eb21f619917 (table default.Table_d2f34f53adf549cc97c5de72f7805087 [id=9f15fd21f99640c7b59422028da7b8c1]) successfully deleted
I20260320 15:27:36.313956 28734 raft_consensus.cc:359] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.314146 28734 raft_consensus.cc:385] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.314200 28734 raft_consensus.cc:740] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.314389 28734 consensus_queue.cc:260] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.314987 28734 ts_tablet_manager.cc:1434] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:36.314038 28444 raft_consensus.cc:2468] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.315333 28734 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.317401 28734 tablet_bootstrap.cc:654] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.312124 29840 raft_consensus.cc:2749] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.323217 29640 tablet_replica.cc:333] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.323660 29640 raft_consensus.cc:2243] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.323869 29640 raft_consensus.cc:2272] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.328325 29640 ts_tablet_manager.cc:1916] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.333143 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:36.343083 28733 tablet_bootstrap.cc:654] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.343600 29886 raft_consensus.cc:493] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.343761 29886 raft_consensus.cc:515] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.344405 29886 leader_election.cc:290] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.345139 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.345376 28583 raft_consensus.cc:2468] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:36.346010 28382 leader_election.cc:304] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.346375 29886 raft_consensus.cc:2804] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.346449 29886 raft_consensus.cc:493] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.346508 29886 raft_consensus.cc:3060] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.329550 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
W20260320 15:27:36.353367 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 45017.6us
I20260320 15:27:36.361964 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.362193 28302 raft_consensus.cc:2468] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:36.364670 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:36.364916 28736 tablet_bootstrap.cc:492] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.365135 28736 ts_tablet_manager.cc:1403] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.081s user 0.004s sys 0.007s
I20260320 15:27:36.360659 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4c6f8c20b5904eafb9a62d14bb155c82 (table default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69]). Expected version 2 got 0
I20260320 15:27:36.368171 28736 raft_consensus.cc:359] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.369107 28736 raft_consensus.cc:385] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.369372 28736 raft_consensus.cc:740] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.370188 28736 consensus_queue.cc:260] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.373440 28736 ts_tablet_manager.cc:1434] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260320 15:27:36.376124 28736 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.376917 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
W20260320 15:27:36.389860 29807 log.cc:927] Time spent T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Append to log took a long time: real 0.089s user 0.002s sys 0.000s
I20260320 15:27:36.391083 28736 tablet_bootstrap.cc:654] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.394009 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:36.402339 29842 raft_consensus.cc:515] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.403082 29842 leader_election.cc:290] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.403424 29676 raft_consensus.cc:515] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.404078 29676 leader_election.cc:290] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.406213 28297 raft_consensus.cc:2468] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.409444 29676 raft_consensus.cc:493] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.409621 29676 raft_consensus.cc:515] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.410317 29676 leader_election.cc:290] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.411350 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:36.411576 28444 raft_consensus.cc:3060] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.412030 29678 raft_consensus.cc:2272] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.412812 29678 ts_tablet_manager.cc:1916] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.414496 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26818e620ff649d18b671e4f4c7c7e74" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.414707 28583 raft_consensus.cc:3060] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.418448 28736 tablet_bootstrap.cc:492] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.418812 28736 ts_tablet_manager.cc:1403] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.002s
I20260320 15:27:36.419322 28733 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.419555 28733 ts_tablet_manager.cc:1403] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.125s user 0.009s sys 0.008s
I20260320 15:27:36.419591 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.419896 28443 raft_consensus.cc:2393] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:36.420569 28733 raft_consensus.cc:359] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.420871 28733 raft_consensus.cc:385] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.420974 28733 raft_consensus.cc:740] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.421293 28733 consensus_queue.cc:260] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.421154 28736 raft_consensus.cc:359] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.421646 28736 raft_consensus.cc:385] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.421797 28736 raft_consensus.cc:740] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.423050 28736 consensus_queue.cc:260] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.425128 28736 ts_tablet_manager.cc:1434] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:36.426359 28736 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.430047 28733 ts_tablet_manager.cc:1434] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:36.430564 28733 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.432463 28733 tablet_bootstrap.cc:654] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.433058 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecaa08ba34514c5e99a9e2d4ba9268a9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.433259 28297 raft_consensus.cc:3060] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.433306 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4cad900411e4c9686701aa99206dd23" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.433669 28736 tablet_bootstrap.cc:654] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.441237 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26818e620ff649d18b671e4f4c7c7e74" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.441501 28302 raft_consensus.cc:3060] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.448925 29887 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.449208 29887 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.451498 29887 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.455703 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.457402 28736 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.457929 28736 ts_tablet_manager.cc:1403] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.004s
W20260320 15:27:36.457712 28383 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.459272 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:36.433966 28524 leader_election.cc:304] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.450850 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.464383 29676 raft_consensus.cc:3060] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:36.464433 28382 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.464574 28382 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.465025 29842 raft_consensus.cc:2749] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.465981 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:36.470079 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:36.471519 29842 raft_consensus.cc:493] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.471702 29842 raft_consensus.cc:515] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.472470 29842 leader_election.cc:290] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.472828 29599 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.472985 29599 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.473449 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4eb20eb0f1403282030727757031f7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.473623 29599 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.473846 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4eb20eb0f1403282030727757031f7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.474001 28575 raft_consensus.cc:2468] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
W20260320 15:27:36.474545 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:36.474726 28382 leader_election.cc:304] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.475214 29842 raft_consensus.cc:2804] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.475327 29842 raft_consensus.cc:493] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.475400 29842 raft_consensus.cc:3060] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.479597 28733 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.479803 28733 ts_tablet_manager.cc:1403] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.003s
I20260320 15:27:36.480407 28734 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.480569 28734 ts_tablet_manager.cc:1403] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.165s user 0.039s sys 0.028s
I20260320 15:27:36.481006 28733 raft_consensus.cc:359] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.481204 28733 raft_consensus.cc:385] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.481274 28733 raft_consensus.cc:740] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.481526 28733 consensus_queue.cc:260] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.481633 28734 raft_consensus.cc:359] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.481815 28734 raft_consensus.cc:385] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.481868 28734 raft_consensus.cc:740] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.482126 28733 ts_tablet_manager.cc:1434] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:36.482070 28734 consensus_queue.cc:260] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.482455 28733 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.482795 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:36.484040 28734 ts_tablet_manager.cc:1434] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:36.484359 28734 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.484404 28733 tablet_bootstrap.cc:654] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.490898 28734 tablet_bootstrap.cc:654] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.490049 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:36.494585 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:36.500661 29886 raft_consensus.cc:515] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.501377 29886 leader_election.cc:290] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.504209 28444 raft_consensus.cc:2468] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.504753 29640 ts_tablet_manager.cc:1929] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.504876 29640 log.cc:1199] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:36.505141 28734 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.505281 29640 ts_tablet_manager.cc:1950] T 1c2d7e4d26994a17bf2d2615d6552412 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.505283 28734 ts_tablet_manager.cc:1403] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20260320 15:27:36.505600 29555 raft_consensus.cc:2749] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.506084 29643 ts_tablet_manager.cc:1929] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.506256 29643 log.cc:1199] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5bc3fb47136d4b88a2898af399d74a8a
I20260320 15:27:36.506228 28734 raft_consensus.cc:359] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.506461 28734 raft_consensus.cc:385] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.506531 28734 raft_consensus.cc:740] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.506650 29643 ts_tablet_manager.cc:1950] T 5bc3fb47136d4b88a2898af399d74a8a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:36.506747 28734 consensus_queue.cc:260] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.507038 28583 raft_consensus.cc:2468] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.507772 28734 ts_tablet_manager.cc:1434] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:36.508275 28734 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.508608 28382 leader_election.cc:304] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.509611 29835 raft_consensus.cc:2804] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.509728 29835 raft_consensus.cc:697] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.510010 28734 tablet_bootstrap.cc:654] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.509954 29835 consensus_queue.cc:237] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.511088 28524 leader_election.cc:304] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.511204 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.511372 28583 raft_consensus.cc:3060] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.511394 29599 raft_consensus.cc:2804] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.511549 29599 raft_consensus.cc:697] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:36.511806 29599 consensus_queue.cc:237] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.529193 29887 raft_consensus.cc:493] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.529376 29887 raft_consensus.cc:515] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.529891 28734 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.530028 29887 leader_election.cc:290] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:36.530059 28734 ts_tablet_manager.cc:1403] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.000s
I20260320 15:27:36.531234 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.531468 28734 raft_consensus.cc:359] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.531683 28734 raft_consensus.cc:385] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.531749 28734 raft_consensus.cc:740] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.531980 28734 consensus_queue.cc:260] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.532626 28734 ts_tablet_manager.cc:1434] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:36.532938 28734 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.533854 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:36.534996 28734 tablet_bootstrap.cc:654] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.531466 28575 raft_consensus.cc:2468] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:36.536834 28382 leader_election.cc:304] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.537119 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ee4a0d559ce4dfea878bd3f1c03c09a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.537164 28736 raft_consensus.cc:359] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.537252 29886 raft_consensus.cc:2804] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.537335 29886 raft_consensus.cc:493] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.537307 28299 raft_consensus.cc:3060] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.537307 28736 raft_consensus.cc:385] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.537408 29886 raft_consensus.cc:3060] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.537433 28736 raft_consensus.cc:740] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.537640 28736 consensus_queue.cc:260] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.538112 28736 ts_tablet_manager.cc:1434] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.080s user 0.001s sys 0.001s
I20260320 15:27:36.538861 28736 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.540135 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:36.540992 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:36.541132 28524 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.541534 29684 raft_consensus.cc:2749] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:36.549366 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
W20260320 15:27:36.555147 28383 leader_election.cc:343] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:36.556138 28383 leader_election.cc:343] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.555589 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:36.560527 28734 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.560684 28734 ts_tablet_manager.cc:1403] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.004s
I20260320 15:27:36.561497 28734 raft_consensus.cc:359] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.561679 28734 raft_consensus.cc:385] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.561738 28734 raft_consensus.cc:740] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.561956 28734 consensus_queue.cc:260] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.556178 29835 raft_consensus.cc:493] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.562520 28734 ts_tablet_manager.cc:1434] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:36.562806 28734 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.562623 29835 raft_consensus.cc:515] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.564392 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.564668 28734 tablet_bootstrap.cc:654] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:36.564963 28382 leader_election.cc:343] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.565524 29835 leader_election.cc:290] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.566049 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:36.566725 28383 leader_election.cc:343] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.566919 28383 leader_election.cc:304] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.567385 29835 raft_consensus.cc:2749] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.569401 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:36.574438 28734 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.574793 28734 ts_tablet_manager.cc:1403] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260320 15:27:36.574594 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:36.575971 28734 raft_consensus.cc:359] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.576174 28734 raft_consensus.cc:385] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.575845 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:36.576241 28734 raft_consensus.cc:740] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.576552 28734 consensus_queue.cc:260] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.576790 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:36.577143 28734 ts_tablet_manager.cc:1434] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260320 15:27:36.577512 28734 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.577986 28736 tablet_bootstrap.cc:654] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.577900 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_ec3f886f78ed4f3092a8d4e6cf282475 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 3 got 0
I20260320 15:27:36.579594 28734 tablet_bootstrap.cc:654] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.579469 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:36.583022 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:36.583498 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:36.584148 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:36.585146 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:36.586174 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:36.586808 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:36.587602 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:36.594714 29886 raft_consensus.cc:515] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.596156 28299 raft_consensus.cc:2468] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.596901 29676 raft_consensus.cc:2749] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.597048 28383 leader_election.cc:304] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.597391 29835 raft_consensus.cc:2804] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.597393 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 82) took 2966 ms. Trace:
I20260320 15:27:36.597512 29835 raft_consensus.cc:697] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.597777 29835 consensus_queue.cc:237] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.600858 29886 leader_election.cc:290] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:36.600816 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:36.602502 28383 leader_election.cc:343] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.604585 28736 tablet_bootstrap.cc:492] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.604988 28736 ts_tablet_manager.cc:1403] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.066s user 0.005s sys 0.008s
I20260320 15:27:36.606693 28736 raft_consensus.cc:359] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.607118 28736 raft_consensus.cc:385] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.607201 28736 raft_consensus.cc:740] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.607604 28736 consensus_queue.cc:260] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.608664 28736 ts_tablet_manager.cc:1434] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:36.609241 28734 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.609395 28734 ts_tablet_manager.cc:1403] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.015s
I20260320 15:27:36.609558 28736 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.610234 28734 raft_consensus.cc:359] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.610420 28734 raft_consensus.cc:385] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.610479 28734 raft_consensus.cc:740] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.610697 28734 consensus_queue.cc:260] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.611279 28734 ts_tablet_manager.cc:1434] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:36.612059 29842 raft_consensus.cc:515] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.612370 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 88) took 2622 ms. Trace:
I20260320 15:27:36.612452 29643 rpcz_store.cc:276] 0320 15:27:33.989740 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.989835 (+ 95us) service_pool.cc:225] Handling call
0320 15:27:36.309601 (+2319766us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:36.309607 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.402303 (+ 92696us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.505967 (+103664us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.612351 (+106384us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2319418}
I20260320 15:27:36.612797 29842 leader_election.cc:290] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.613000 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 89) took 2620 ms. Trace:
I20260320 15:27:36.613086 29643 rpcz_store.cc:276] 0320 15:27:33.992841 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.993162 (+ 321us) service_pool.cc:225] Handling call
0320 15:27:36.612796 (+2619634us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:36.612807 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.612983 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2619264}
I20260320 15:27:36.613332 29643 tablet_replica.cc:333] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:36.613550 29643 raft_consensus.cc:2243] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.613730 29643 raft_consensus.cc:2272] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.614331 29643 ts_tablet_manager.cc:1916] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.614593 28736 tablet_bootstrap.cc:654] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.618892 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4eb20eb0f1403282030727757031f7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.619143 28298 raft_consensus.cc:3060] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.620179 29678 ts_tablet_manager.cc:1929] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.620309 29678 log.cc:1199] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2165371f4b5448c7a2625926082ee4c6
I20260320 15:27:36.596303 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.620671 28575 raft_consensus.cc:3060] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.620733 29678 ts_tablet_manager.cc:1950] T 2165371f4b5448c7a2625926082ee4c6 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:36.616434 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba4eb20eb0f1403282030727757031f7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.621075 28584 raft_consensus.cc:3060] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.597476 29640 rpcz_store.cc:276] 0320 15:27:33.630660 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.630766 (+ 106us) service_pool.cc:225] Handling call
0320 15:27:36.323146 (+2692380us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:36.323168 (+ 22us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.404363 (+ 81195us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.504609 (+100246us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.597375 (+ 92766us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2645420}
I20260320 15:27:36.621842 29640 tablet_replica.cc:333] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.622097 29640 raft_consensus.cc:2243] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.622977 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 1c2d7e4d26994a17bf2d2615d6552412 (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]) successfully deleted
I20260320 15:27:36.625808 29640 raft_consensus.cc:2272] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.626698 29640 ts_tablet_manager.cc:1916] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.628151 28734 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.630056 28734 tablet_bootstrap.cc:654] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.630733 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 5bc3fb47136d4b88a2898af399d74a8a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:36.631058 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7a4b5a658cd84b04b50b54973c065889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a4b5a658cd84b04b50b54973c065889 already in progress: creating tablet
I20260320 15:27:36.634918 29676 raft_consensus.cc:493] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.635131 29676 raft_consensus.cc:515] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.635876 29676 leader_election.cc:290] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:36.637145 28524 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 66a9ddea49e641da881fba339ddef549
W20260320 15:27:36.637578 28524 leader_election.cc:343] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 66a9ddea49e641da881fba339ddef549
I20260320 15:27:36.637677 28524 leader_election.cc:304] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.638108 29676 raft_consensus.cc:2749] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:36.646299 28109 leader_election.cc:336] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Timed out: RequestConsensusVote RPC to 127.27.74.61:44739 timed out after 0.640s (SENT)
I20260320 15:27:36.646502 28109 leader_election.cc:304] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 336af1a126ca4ab583c03527b890239a; no voters: 07e8bb8819eb4cb4892e0cba1f758892, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:36.649878 28733 tablet_bootstrap.cc:492] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.650415 28733 ts_tablet_manager.cc:1403] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.168s user 0.006s sys 0.001s
I20260320 15:27:36.677089 29950 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 336af1a126ca4ab583c03527b890239a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260320 15:27:36.686043 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:36.686623 28583 raft_consensus.cc:2468] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.686700 28734 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.686831 28734 ts_tablet_manager.cc:1403] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.059s user 0.004s sys 0.003s
I20260320 15:27:36.687568 28302 raft_consensus.cc:2468] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.687706 28734 raft_consensus.cc:359] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.687873 28734 raft_consensus.cc:385] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.687916 28734 raft_consensus.cc:740] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.688097 28734 consensus_queue.cc:260] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.688570 28734 ts_tablet_manager.cc:1434] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:36.688834 28734 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.690511 28734 tablet_bootstrap.cc:654] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.691216 28297 raft_consensus.cc:2468] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.693089 28575 raft_consensus.cc:2468] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.694372 28298 raft_consensus.cc:2468] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.694905 28584 raft_consensus.cc:2468] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.695709 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 90) took 2757 ms. Trace:
I20260320 15:27:36.695804 29678 rpcz_store.cc:276] 0320 15:27:33.937888 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.938082 (+ 194us) service_pool.cc:225] Handling call
0320 15:27:36.205156 (+2267074us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:36.205166 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.527715 (+322549us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.620040 (+ 92325us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.695691 (+ 75651us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2266545}
I20260320 15:27:36.695920 28382 leader_election.cc:304] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:36.696156 29678 tablet_replica.cc:333] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:36.696364 29678 raft_consensus.cc:2243] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.696412 29842 raft_consensus.cc:2804] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.696549 29842 raft_consensus.cc:697] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.696691 29678 raft_consensus.cc:2272] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.696812 29842 consensus_queue.cc:237] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.696321 28733 raft_consensus.cc:359] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.697292 29678 ts_tablet_manager.cc:1916] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.697610 28733 raft_consensus.cc:385] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.698027 28733 raft_consensus.cc:740] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.698827 28383 leader_election.cc:304] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.699107 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 2165371f4b5448c7a2625926082ee4c6 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
I20260320 15:27:36.699157 29835 raft_consensus.cc:2804] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.699282 29835 raft_consensus.cc:697] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.699514 29835 consensus_queue.cc:237] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.699427 28733 consensus_queue.cc:260] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.703307 28733 ts_tablet_manager.cc:1434] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.052s user 0.000s sys 0.003s
I20260320 15:27:36.706115 28733 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.710400 29886 raft_consensus.cc:493] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.710589 29886 raft_consensus.cc:515] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.711303 29886 leader_election.cc:290] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.711939 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:36.712563 28383 leader_election.cc:343] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.713398 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:36.715837 28382 leader_election.cc:343] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.715998 28382 leader_election.cc:304] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.716964 29640 ts_tablet_manager.cc:1929] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.717111 29640 log.cc:1199] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/757b5afdea674217909610e3719a8804
I20260320 15:27:36.717581 29640 ts_tablet_manager.cc:1950] T 757b5afdea674217909610e3719a8804 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.720331 29835 raft_consensus.cc:2749] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.728632 28733 tablet_bootstrap.cc:654] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.738925 29643 ts_tablet_manager.cc:1929] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.739077 29643 log.cc:1199] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:36.739497 29643 ts_tablet_manager.cc:1950] T 931f6f88a1d94a40b69f07216b8313c8 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:36.739929 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 90) took 2726 ms. Trace:
I20260320 15:27:36.740013 29643 rpcz_store.cc:276] 0320 15:27:34.013775 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.013845 (+ 70us) service_pool.cc:225] Handling call
0320 15:27:36.613292 (+2599447us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:36.613301 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.687985 (+ 74684us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.738792 (+ 50807us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.739904 (+ 1112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2599155}
I20260320 15:27:36.740371 29643 tablet_replica.cc:333] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:36.740595 29643 raft_consensus.cc:2243] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.740651 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 931f6f88a1d94a40b69f07216b8313c8 (table default.Table_e25b7a014de24f8c8f22a3584fd0d367 [id=19af05ab6c6544b99f6b348f2c52b62b]) successfully deleted
I20260320 15:27:36.740773 29643 raft_consensus.cc:2272] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.741401 29643 ts_tablet_manager.cc:1916] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.744380 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 83) took 3105 ms. Trace:
I20260320 15:27:36.744508 29640 rpcz_store.cc:276] 0320 15:27:33.673587 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.673709 (+ 122us) service_pool.cc:225] Handling call
0320 15:27:36.621768 (+2948059us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:36.621786 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.687055 (+ 65269us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.716791 (+ 29736us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.744346 (+ 27555us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2937795}
I20260320 15:27:36.744916 29640 tablet_replica.cc:333] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.745138 29640 raft_consensus.cc:2243] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.745311 29640 raft_consensus.cc:2272] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.746001 29640 ts_tablet_manager.cc:1916] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.747148 29678 ts_tablet_manager.cc:1929] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.747269 29678 log.cc:1199] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:36.748013 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 757b5afdea674217909610e3719a8804 (table default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]) successfully deleted
W20260320 15:27:36.760522 28872 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4190329df1b644c392ff8cbace9aae7a consensus state has no leader
I20260320 15:27:36.766896 29678 ts_tablet_manager.cc:1950] T b82c47ae3786401d931bb0b486ca3570 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:36.767333 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 91) took 2818 ms. Trace:
I20260320 15:27:36.767432 29678 rpcz_store.cc:276] 0320 15:27:33.948735 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.948884 (+ 149us) service_pool.cc:225] Handling call
0320 15:27:36.696101 (+2747217us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:36.696113 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.743279 (+ 47166us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.747010 (+ 3731us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.767313 (+ 20303us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2746729}
I20260320 15:27:36.768208 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b82c47ae3786401d931bb0b486ca3570 (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
I20260320 15:27:36.768709 29678 tablet_replica.cc:333] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:36.768959 29678 raft_consensus.cc:2243] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.769505 29678 raft_consensus.cc:2272] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.770340 29678 ts_tablet_manager.cc:1916] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.781731 29643 ts_tablet_manager.cc:1929] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.785615 29640 ts_tablet_manager.cc:1929] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.795687 29643 log.cc:1199] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:36.796372 29643 ts_tablet_manager.cc:1950] T 993e2fb1561f4317be85e9b5e5816edc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:36.796566 29640 log.cc:1199] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:36.797037 29640 ts_tablet_manager.cc:1950] T f5f92561764b4cb9b15bb9a310a350f0 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.797536 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 91) took 2778 ms. Trace:
I20260320 15:27:36.798008 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 84) took 3099 ms. Trace:
I20260320 15:27:36.798931 28736 tablet_bootstrap.cc:492] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.799106 28736 ts_tablet_manager.cc:1403] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.190s user 0.004s sys 0.021s
I20260320 15:27:36.799573 29640 rpcz_store.cc:276] 0320 15:27:33.698713 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.699018 (+ 305us) service_pool.cc:225] Handling call
0320 15:27:36.744860 (+3045842us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:36.744874 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.762407 (+ 17533us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.785466 (+ 23059us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.797989 (+ 12523us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3043329}
I20260320 15:27:36.798192 29643 rpcz_store.cc:276] 0320 15:27:34.019088 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.019169 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:36.740327 (+2721158us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:36.740337 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.760161 (+ 19824us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.781544 (+ 21383us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.797473 (+ 15929us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2720813}
I20260320 15:27:36.800107 28736 raft_consensus.cc:359] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.800313 28736 raft_consensus.cc:385] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.800379 28736 raft_consensus.cc:740] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.800650 28736 consensus_queue.cc:260] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.800815 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 993e2fb1561f4317be85e9b5e5816edc (table default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]) successfully deleted
I20260320 15:27:36.801249 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet f5f92561764b4cb9b15bb9a310a350f0 (table default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]) successfully deleted
I20260320 15:27:36.800279 29643 tablet_replica.cc:333] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:36.801728 29640 tablet_replica.cc:333] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.802031 29643 raft_consensus.cc:2243] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.804028 28736 ts_tablet_manager.cc:1434] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:36.804347 28736 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.805756 29555 raft_consensus.cc:493] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.805917 29555 raft_consensus.cc:515] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.806653 29555 leader_election.cc:290] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.807996 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.808204 28584 raft_consensus.cc:2468] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:36.808694 28239 leader_election.cc:304] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:36.809099 29555 raft_consensus.cc:2804] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.809181 29555 raft_consensus.cc:493] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.809244 29555 raft_consensus.cc:3060] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.812318 29555 raft_consensus.cc:515] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.813494 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.813670 28584 raft_consensus.cc:3060] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.816526 28584 raft_consensus.cc:2468] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:36.818118 28733 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.818171 28239 leader_election.cc:304] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:36.818405 28733 ts_tablet_manager.cc:1403] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.007s
I20260320 15:27:36.818468 29618 raft_consensus.cc:2804] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.818590 29618 raft_consensus.cc:697] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:36.818832 29618 consensus_queue.cc:237] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.819360 28733 raft_consensus.cc:359] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.819621 28733 raft_consensus.cc:385] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.819727 28733 raft_consensus.cc:740] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:36.820062 28524 consensus_peers.cc:597] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c752fb99ef494bda9d144d0e180e3100. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:36.820067 28733 consensus_queue.cc:260] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.821812 28733 ts_tablet_manager.cc:1434] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:36.822059 29555 leader_election.cc:290] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.823180 29678 ts_tablet_manager.cc:1929] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.823299 29678 log.cc:1199] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b1bc86d3d1ff4b3aa8463eb21f619917
I20260320 15:27:36.823407 28733 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.825245 28733 tablet_bootstrap.cc:654] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.826598 28736 tablet_bootstrap.cc:654] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.807363 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.834316 28444 raft_consensus.cc:2468] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:36.819633 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43a0fadc7dac4e21ad034e4cd2d70826" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:36.820971 28734 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.835204 28443 raft_consensus.cc:3060] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.835345 28734 ts_tablet_manager.cc:1403] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.147s user 0.000s sys 0.026s
I20260320 15:27:36.836267 28734 raft_consensus.cc:359] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.836484 28734 raft_consensus.cc:385] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.836561 28734 raft_consensus.cc:740] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.836846 28734 consensus_queue.cc:260] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.837416 28734 ts_tablet_manager.cc:1434] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:36.838608 28734 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.839085 28443 raft_consensus.cc:2468] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:36.839640 28736 tablet_bootstrap.cc:492] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.839787 28736 ts_tablet_manager.cc:1403] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.001s
I20260320 15:27:36.840134 29643 raft_consensus.cc:2272] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.840581 28736 raft_consensus.cc:359] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.840762 28736 raft_consensus.cc:385] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.840823 28736 raft_consensus.cc:740] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.841171 28736 consensus_queue.cc:260] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.841816 28736 ts_tablet_manager.cc:1434] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:36.841811 29643 ts_tablet_manager.cc:1916] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.842125 28736 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.827590 29939 raft_consensus.cc:493] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.842875 29939 raft_consensus.cc:515] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.843458 29939 leader_election.cc:290] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.842708 28734 tablet_bootstrap.cc:654] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.831840 29640 raft_consensus.cc:2243] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:36.845202 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:36.845575 29640 raft_consensus.cc:2272] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:36.846342 28524 leader_election.cc:343] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:36.850610 28524 leader_election.cc:304] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:36.850736 29640 ts_tablet_manager.cc:1916] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.851297 29939 raft_consensus.cc:2749] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.853917 28736 tablet_bootstrap.cc:654] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.862032 29678 ts_tablet_manager.cc:1950] T b1bc86d3d1ff4b3aa8463eb21f619917 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:36.863083 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 87) took 2932 ms. Trace:
I20260320 15:27:36.863256 29678 rpcz_store.cc:276] 0320 15:27:33.930598 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.931585 (+ 987us) service_pool.cc:225] Handling call
0320 15:27:36.768656 (+2837071us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:36.768670 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.819695 (+ 51025us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.823072 (+ 3377us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.863054 (+ 39982us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2809063}
I20260320 15:27:36.863674 29678 tablet_replica.cc:333] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:36.863888 29678 raft_consensus.cc:2243] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.863896 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b1bc86d3d1ff4b3aa8463eb21f619917 (table default.Table_d2f34f53adf549cc97c5de72f7805087 [id=9f15fd21f99640c7b59422028da7b8c1]) successfully deleted
I20260320 15:27:36.864269 29678 raft_consensus.cc:2272] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.864884 29678 ts_tablet_manager.cc:1916] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.869354 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:36.872756 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:36.875715 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:36.876536 28733 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.876830 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:36.877106 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:36.877336 28733 ts_tablet_manager.cc:1403] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.055s user 0.005s sys 0.005s
I20260320 15:27:36.877790 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:36.881748 28736 tablet_bootstrap.cc:492] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.881906 28736 ts_tablet_manager.cc:1403] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.001s
I20260320 15:27:36.882683 28736 raft_consensus.cc:359] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.882683 28733 raft_consensus.cc:359] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.882866 28736 raft_consensus.cc:385] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.882920 28736 raft_consensus.cc:740] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.883111 28736 consensus_queue.cc:260] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.883966 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:36.884457 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:36.884639 28733 raft_consensus.cc:385] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.884771 28733 raft_consensus.cc:740] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.883656 29640 ts_tablet_manager.cc:1929] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.885079 29640 log.cc:1199] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:36.885584 29640 ts_tablet_manager.cc:1950] T a4bfb87d5d5741adb0ad2f2a88f4867f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.886263 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:36.886795 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 85) took 3186 ms. Trace:
I20260320 15:27:36.887591 29640 rpcz_store.cc:276] 0320 15:27:33.699931 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.700042 (+ 111us) service_pool.cc:225] Handling call
0320 15:27:36.801676 (+3101634us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:36.801688 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.864174 (+ 62486us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.883505 (+ 19331us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.886774 (+ 3269us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3099945}
I20260320 15:27:36.885787 28733 consensus_queue.cc:260] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.888288 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet a4bfb87d5d5741adb0ad2f2a88f4867f (table default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]) successfully deleted
I20260320 15:27:36.889003 28736 ts_tablet_manager.cc:1434] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:36.898196 29643 ts_tablet_manager.cc:1929] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.898394 29643 log.cc:1199] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2165371f4b5448c7a2625926082ee4c6
I20260320 15:27:36.898732 29678 ts_tablet_manager.cc:1929] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.899181 28734 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.899365 28734 ts_tablet_manager.cc:1403] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.002s
I20260320 15:27:36.899555 29678 log.cc:1199] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:36.900004 29678 ts_tablet_manager.cc:1950] T 931f6f88a1d94a40b69f07216b8313c8 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:36.901047 28734 raft_consensus.cc:359] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.901230 29842 raft_consensus.cc:493] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.901335 28734 raft_consensus.cc:385] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.901420 29842 raft_consensus.cc:515] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.901638 28734 raft_consensus.cc:740] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.901872 28734 consensus_queue.cc:260] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.902118 29842 leader_election.cc:290] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.902427 28734 ts_tablet_manager.cc:1434] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:36.902750 28734 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.902840 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.903664 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:36.904353 28382 leader_election.cc:343] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.904772 28734 tablet_bootstrap.cc:654] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.905326 28736 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:36.906711 28383 leader_election.cc:343] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.906860 28383 leader_election.cc:304] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.907271 29835 raft_consensus.cc:2749] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.907883 28736 tablet_bootstrap.cc:654] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.909894 28733 ts_tablet_manager.cc:1434] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.028s user 0.001s sys 0.001s
W20260320 15:27:36.910312 28240 consensus_peers.cc:597] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5bc3fb47136d4b88a2898af399d74a8a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:36.915874 28736 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:36.916023 28736 ts_tablet_manager.cc:1403] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20260320 15:27:36.916788 28736 raft_consensus.cc:359] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.916953 28736 raft_consensus.cc:385] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.917002 28736 raft_consensus.cc:740] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.917198 28736 consensus_queue.cc:260] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.918309 28736 ts_tablet_manager.cc:1434] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:36.918627 28736 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:36.920406 28736 tablet_bootstrap.cc:654] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.920455 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 86) took 2990 ms. Trace:
I20260320 15:27:36.920573 29678 rpcz_store.cc:276] 0320 15:27:33.930163 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.930325 (+ 162us) service_pool.cc:225] Handling call
0320 15:27:36.863627 (+2933302us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:36.863636 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.892034 (+ 28398us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.898611 (+ 6577us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.920431 (+ 21820us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2903538}
I20260320 15:27:36.921185 29678 tablet_replica.cc:333] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:36.921175 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 931f6f88a1d94a40b69f07216b8313c8 (table default.Table_e25b7a014de24f8c8f22a3584fd0d367 [id=19af05ab6c6544b99f6b348f2c52b62b]) successfully deleted
I20260320 15:27:36.922009 28733 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.922946 29640 tablet_replica.cc:333] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.923215 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:36.923521 29640 raft_consensus.cc:2243] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:36.924096 29640 raft_consensus.cc:2272] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.924733 29640 ts_tablet_manager.cc:1916] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.926766 29835 raft_consensus.cc:493] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.926936 29835 raft_consensus.cc:515] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.927652 29835 leader_election.cc:290] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.928407 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:36.928658 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3f6d9a528946908891f77f951d23dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.928778 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:36.928817 29676 raft_consensus.cc:493] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.928849 28584 raft_consensus.cc:2468] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:36.928946 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:36.928936 29676 raft_consensus.cc:515] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.929430 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:36.929431 28382 leader_election.cc:304] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.929618 29676 leader_election.cc:290] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.929738 29835 raft_consensus.cc:2804] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.929791 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:36.929857 29835 raft_consensus.cc:493] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.929956 29835 raft_consensus.cc:3060] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.930401 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:36.930649 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:36.930837 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:36.930972 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.931212 28443 raft_consensus.cc:2468] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.931941 28524 leader_election.cc:304] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.932089 28733 tablet_bootstrap.cc:654] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.932407 29599 raft_consensus.cc:2804] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.932493 29599 raft_consensus.cc:493] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.932559 29599 raft_consensus.cc:3060] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.932857 29835 raft_consensus.cc:515] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.933789 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3f6d9a528946908891f77f951d23dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:36.933935 28584 raft_consensus.cc:3060] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.935174 29599 raft_consensus.cc:515] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.936025 29599 leader_election.cc:290] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.936558 28584 raft_consensus.cc:2468] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:36.936559 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:36.937114 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:36.937160 28382 leader_election.cc:304] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.937389 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:36.937623 28443 raft_consensus.cc:3060] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.938261 29835 leader_election.cc:290] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.938477 29835 raft_consensus.cc:2804] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.938890 28734 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:36.939008 29835 raft_consensus.cc:697] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:36.939060 28734 ts_tablet_manager.cc:1403] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260320 15:27:36.939337 29835 consensus_queue.cc:237] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.941769 28443 raft_consensus.cc:2468] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.943110 28524 leader_election.cc:304] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:36.943574 29842 raft_consensus.cc:493] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.943727 29842 raft_consensus.cc:515] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.943915 29886 consensus_queue.cc:1048] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:36.944044 29599 raft_consensus.cc:2804] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.944392 29842 leader_election.cc:290] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:36.944427 29599 raft_consensus.cc:697] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:36.945091 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.945473 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
W20260320 15:27:36.945706 28382 leader_election.cc:343] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.946266 29599 consensus_queue.cc:237] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:36.947610 28734 raft_consensus.cc:359] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.947901 28734 raft_consensus.cc:385] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.947993 28734 raft_consensus.cc:740] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.948243 28734 consensus_queue.cc:260] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:36.948836 28734 ts_tablet_manager.cc:1434] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:36.949162 28734 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:36.950996 28734 tablet_bootstrap.cc:654] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.952580 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:36.953052 28733 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:36.953153 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ca5e0c215f574b8aafec8d5e2da9a399 (table default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2]). Expected version 3 got 0
I20260320 15:27:36.953210 28733 ts_tablet_manager.cc:1403] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.001s
I20260320 15:27:36.953363 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:36.953948 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:36.954041 28733 raft_consensus.cc:359] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.954157 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:36.954231 28733 raft_consensus.cc:385] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:36.954290 28733 raft_consensus.cc:740] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:36.954501 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:36.954511 28733 consensus_queue.cc:260] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.954849 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:36.955240 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:36.928648 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3f6d9a528946908891f77f951d23dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.930384 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.957755 28297 raft_consensus.cc:2468] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.930548 29618 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:36.958020 28383 leader_election.cc:343] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.958029 29618 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.958740 29618 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.936909 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "583ed053b2a742c7b029f348f4495101" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.958935 28302 raft_consensus.cc:3060] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.939896 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3f6d9a528946908891f77f951d23dd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.944893 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.961598 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:36.961840 29640 ts_tablet_manager.cc:1929] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:36.961900 28733 ts_tablet_manager.cc:1434] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
W20260320 15:27:36.962172 28239 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.962688 28302 raft_consensus.cc:2468] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.961954 29640 log.cc:1199] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:36.965095 29640 ts_tablet_manager.cc:1950] T 9aa2d48a54dd403c82d7f3a64db5aeab P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:36.966120 29678 raft_consensus.cc:2243] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.966557 29678 raft_consensus.cc:2272] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.967373 29678 ts_tablet_manager.cc:1916] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:36.968276 28383 leader_election.cc:343] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:36.968743 28383 leader_election.cc:343] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:36.968863 28383 leader_election.cc:304] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.969003 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 86) took 3257 ms. Trace:
I20260320 15:27:36.969080 29640 rpcz_store.cc:276] 0320 15:27:33.711751 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.712116 (+ 365us) service_pool.cc:225] Handling call
0320 15:27:36.922899 (+3210783us) ts_tablet_manager.cc:1151] Deleting tablet 9aa2d48a54dd403c82d7f3a64db5aeab
0320 15:27:36.922909 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.941766 (+ 18857us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.961711 (+ 19945us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.968982 (+ 7271us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3210355}
I20260320 15:27:36.969422 29640 tablet_replica.cc:333] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:36.969616 29640 raft_consensus.cc:2243] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:36.969846 29835 raft_consensus.cc:2749] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:36.969945 29640 raft_consensus.cc:2272] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:36.965756 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.970527 29640 ts_tablet_manager.cc:1916] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:36.970613 28444 raft_consensus.cc:2468] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:36.970983 28733 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:36.971468 29676 raft_consensus.cc:493] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:36.971613 29676 raft_consensus.cc:515] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.972127 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:36.972280 29676 leader_election.cc:290] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.972440 28900 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:36.972450 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9aa2d48a54dd403c82d7f3a64db5aeab (table default.Table_469452eb6e4d49b495926119d8ef0568 [id=d3ac714c983c46959909a4653193b33c]) successfully deleted
I20260320 15:27:36.973038 28240 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:36.973119 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:36.973280 28303 raft_consensus.cc:2468] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.973497 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:36.973646 28524 leader_election.cc:304] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:36.973686 28444 raft_consensus.cc:2468] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:36.973996 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 69) took 3440 ms. Trace:
I20260320 15:27:36.974196 29676 raft_consensus.cc:2804] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.974261 29676 raft_consensus.cc:493] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.974321 29676 raft_consensus.cc:3060] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.974131 28906 rpcz_store.cc:276] 0320 15:27:33.533892 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.828766 (+2294874us) service_pool.cc:225] Handling call
0320 15:27:35.829746 (+ 980us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:35.830163 (+ 417us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:35.830605 (+ 442us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:35.830765 (+ 160us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:35.862739 (+ 31974us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:36.973953 (+1111214us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.833702 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.833840 (+ 138us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.833848 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:35.833850 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:35.833966 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.833983 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.833988 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:35.834061 (+ 73us) tablet.cc:700] Row locks acquired
0320 15:27:35.834064 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:35.834104 (+ 40us) write_op.cc:270] Start()
0320 15:27:35.834172 (+ 68us) write_op.cc:276] Timestamp: P: 1774020455834066 usec, L: 0
0320 15:27:35.834176 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.834381 (+ 205us) log.cc:844] Serialized 405 byte log entry
0320 15:27:35.861887 (+ 27506us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.862058 (+ 171us) write_op.cc:301] APPLY: starting
0320 15:27:35.862077 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.862126 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.862222 (+ 96us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.862294 (+ 72us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.862329 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:35.862337 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:35.862443 (+ 106us) log.cc:844] Serialized 23 byte log entry
0320 15:27:35.862520 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.862544 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:35.862551 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.862634 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.862637 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":560,"apply.run_wall_time_us":561,"num_ops":1,"prepare.queue_time_us":1344,"prepare.run_cpu_time_us":781,"prepare.run_wall_time_us":780,"replication_time_us":27673}]]}
I20260320 15:27:36.975611 28733 tablet_bootstrap.cc:654] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:36.977159 29676 raft_consensus.cc:515] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.978250 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:36.978433 28303 raft_consensus.cc:3060] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.981043 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 7f94140a5e8b4a06890bca86456d64a6
I20260320 15:27:36.981598 28905 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 65) took 5598 ms. Trace:
I20260320 15:27:36.981738 28303 raft_consensus.cc:2468] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.981715 28905 rpcz_store.cc:276] 0320 15:27:31.383237 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.669177 (+4285940us) service_pool.cc:225] Handling call
0320 15:27:35.669659 (+ 482us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:35.669817 (+ 158us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:35.669899 (+ 82us) catalog_manager.cc:2824] Locking tablets
0320 15:27:35.711697 (+ 41798us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:35.739405 (+ 27708us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:35.739491 (+ 86us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:36.980948 (+1241457us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:36.981562 (+ 614us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.713736 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.713866 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.713873 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:35.713877 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:35.713996 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.714010 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.714017 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:35.714135 (+ 118us) tablet.cc:700] Row locks acquired
0320 15:27:35.714139 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:35.714173 (+ 34us) write_op.cc:270] Start()
0320 15:27:35.714206 (+ 33us) write_op.cc:276] Timestamp: P: 1774020455714118 usec, L: 0
0320 15:27:35.714212 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.714433 (+ 221us) log.cc:844] Serialized 756 byte log entry
0320 15:27:35.719195 (+ 4762us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.735163 (+ 15968us) write_op.cc:301] APPLY: starting
0320 15:27:35.735177 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.735223 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.735270 (+ 47us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.735339 (+ 69us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.735375 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:35.735381 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:35.735454 (+ 73us) log.cc:844] Serialized 29 byte log entry
0320 15:27:35.735503 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.735523 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:35.735552 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15931,"apply.run_cpu_time_us":460,"apply.run_wall_time_us":460,"num_ops":2,"prepare.queue_time_us":1141,"prepare.run_cpu_time_us":795,"prepare.run_wall_time_us":794,"raft.queue_time_us":1279,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":4965,"thread_start_us":65075,"threads_started":1}]]}
I20260320 15:27:36.982421 28524 leader_election.cc:304] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:36.982765 29939 raft_consensus.cc:2804] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:36.982887 29939 raft_consensus.cc:697] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:36.983134 29939 consensus_queue.cc:237] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:36.984622 29676 leader_election.cc:290] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:36.988749 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a4e61bab44d768d9adfed00c945e9" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:36.988996 28443 raft_consensus.cc:3060] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.991242 29618 raft_consensus.cc:2804] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:36.991353 29618 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:36.991420 29618 raft_consensus.cc:3060] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:36.992112 28443 raft_consensus.cc:2468] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:36.994057 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:36.994769 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:36.995687 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.995849 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.995903 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.995945 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.996367 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.996743 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.996905 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.997022 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.997080 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.997471 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet b395977eb0444bcf8563cd8774fab40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.997663 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:36.997908 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:37.006197 29643 ts_tablet_manager.cc:1950] T 2165371f4b5448c7a2625926082ee4c6 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:37.006750 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 92) took 2984 ms. Trace:
I20260320 15:27:37.006875 29643 rpcz_store.cc:276] 0320 15:27:34.021922 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.022003 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:36.800209 (+2778206us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:36.800220 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:36.888920 (+ 88700us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:36.898040 (+ 9120us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.006686 (+108646us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2777952}
I20260320 15:27:37.007437 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 2165371f4b5448c7a2625926082ee4c6 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
I20260320 15:27:37.011801 29618 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.019114 29886 raft_consensus.cc:493] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.019384 29886 raft_consensus.cc:515] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.020200 29886 leader_election.cc:290] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.021186 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.021615 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.021818 29643 tablet_replica.cc:333] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:37.021940 28382 leader_election.cc:343] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.022064 29643 raft_consensus.cc:2243] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:37.022327 28383 leader_election.cc:343] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.022456 28383 leader_election.cc:304] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.023031 29886 raft_consensus.cc:2749] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.026566 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.040575 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:37.041251 29618 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.043416 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:37.048265 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:37.048635 28443 raft_consensus.cc:3060] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.052875 29643 raft_consensus.cc:2272] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.053726 29643 ts_tablet_manager.cc:1916] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.056186 28733 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.056420 28733 ts_tablet_manager.cc:1403] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.086s user 0.004s sys 0.016s
I20260320 15:27:37.056602 28734 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.056737 28734 ts_tablet_manager.cc:1403] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.108s user 0.003s sys 0.000s
I20260320 15:27:37.057195 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:37.057600 28733 raft_consensus.cc:359] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.057909 28733 raft_consensus.cc:385] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.058022 28733 raft_consensus.cc:740] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.058282 28734 raft_consensus.cc:359] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.058457 28734 raft_consensus.cc:385] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.058509 28734 raft_consensus.cc:740] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.058374 28733 consensus_queue.cc:260] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.058705 28734 consensus_queue.cc:260] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.059170 28734 ts_tablet_manager.cc:1434] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:37.059484 28733 ts_tablet_manager.cc:1434] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:37.061578 28733 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.065310 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:37.066159 28904 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 70) took 3493 ms. Trace:
W20260320 15:27:37.059351 29972 log.cc:927] Time spent T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.094s user 0.000s sys 0.000s
I20260320 15:27:37.066290 28904 rpcz_store.cc:276] 0320 15:27:33.573095 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.905381 (+2332286us) service_pool.cc:225] Handling call
0320 15:27:35.906101 (+ 720us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:35.906315 (+ 214us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:35.906438 (+ 123us) catalog_manager.cc:2824] Locking tablets
0320 15:27:35.910172 (+ 3734us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:35.923902 (+ 13730us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:35.924026 (+ 124us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:37.065247 (+1141221us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:37.066078 (+ 831us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.911427 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.911629 (+ 202us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.911640 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:35.911645 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:35.911818 (+ 173us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.911840 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.911848 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:35.912012 (+ 164us) tablet.cc:700] Row locks acquired
0320 15:27:35.912017 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:35.912072 (+ 55us) write_op.cc:270] Start()
0320 15:27:35.912125 (+ 53us) write_op.cc:276] Timestamp: P: 1774020455912024 usec, L: 0
0320 15:27:35.912131 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.912407 (+ 276us) log.cc:844] Serialized 790 byte log entry
0320 15:27:35.922910 (+ 10503us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.923081 (+ 171us) write_op.cc:301] APPLY: starting
0320 15:27:35.923109 (+ 28us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.923174 (+ 65us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.923258 (+ 84us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.923370 (+ 112us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.923419 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:35.923428 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:35.923567 (+ 139us) log.cc:844] Serialized 29 byte log entry
0320 15:27:35.923645 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.923671 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:35.923679 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.923763 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.923767 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":87,"apply.run_cpu_time_us":670,"apply.run_wall_time_us":669,"num_ops":2,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":1156,"prepare.run_wall_time_us":1157,"raft.queue_time_us":720,"raft.run_cpu_time_us":466,"raft.run_wall_time_us":470,"replication_time_us":10724}]]}
I20260320 15:27:37.068495 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:37.072952 28733 tablet_bootstrap.cc:654] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.075688 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:37.075790 28443 raft_consensus.cc:2468] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:37.076333 28240 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.076752 29960 raft_consensus.cc:2804] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.077025 29960 raft_consensus.cc:697] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:37.077333 29960 consensus_queue.cc:237] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.080369 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:37.081944 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.082360 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.083153 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.083504 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.083853 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.084875 29939 consensus_queue.cc:1048] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.085041 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:37.085624 28239 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.088965 29939 raft_consensus.cc:493] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.089128 29939 raft_consensus.cc:515] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.089751 29939 leader_election.cc:290] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.090337 29886 raft_consensus.cc:493] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.090524 29886 raft_consensus.cc:515] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.091270 29829 raft_consensus.cc:493] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.091430 29829 raft_consensus.cc:515] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.091885 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.092607 28382 leader_election.cc:343] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.092991 29886 leader_election.cc:290] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.093344 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.093456 29829 leader_election.cc:290] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.093567 28303 raft_consensus.cc:2468] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:37.094200 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.094388 28444 raft_consensus.cc:2468] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:37.094885 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.095101 28575 raft_consensus.cc:2468] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:37.095815 28524 leader_election.cc:304] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:37.095999 28240 leader_election.cc:304] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.096124 29939 consensus_queue.cc:1048] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.096261 29829 raft_consensus.cc:2804] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:37.096333 29829 raft_consensus.cc:493] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:37.096356 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.096427 29829 raft_consensus.cc:3060] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.096585 29939 raft_consensus.cc:2804] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:37.096655 29939 raft_consensus.cc:493] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:37.096714 29939 raft_consensus.cc:3060] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:37.097167 28383 leader_election.cc:343] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.097301 28383 leader_election.cc:304] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.097711 29886 raft_consensus.cc:2749] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.099500 29939 raft_consensus.cc:515] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.101121 29939 leader_election.cc:290] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.101312 28736 tablet_bootstrap.cc:492] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.101470 28736 ts_tablet_manager.cc:1403] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.183s user 0.008s sys 0.004s
I20260320 15:27:37.102218 29829 raft_consensus.cc:515] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.103852 29829 leader_election.cc:290] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.103920 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:37.104116 28441 raft_consensus.cc:3060] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.104545 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:37.104745 28446 raft_consensus.cc:3060] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.107236 28441 raft_consensus.cc:2468] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:37.107954 28240 leader_election.cc:304] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.106180 28892 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_7b1d2897e6684a89bb2fe8570e1868d2 [id=cc5e59bc215e444ba6c156b6d0ded84c] alter complete (version 2)
I20260320 15:27:37.108333 29829 raft_consensus.cc:2804] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.108458 29829 raft_consensus.cc:697] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:37.111073 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.112167 28446 raft_consensus.cc:2468] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:37.117142 28524 leader_election.cc:304] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.117650 29939 raft_consensus.cc:2804] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.117753 29835 consensus_queue.cc:1048] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.117987 29939 raft_consensus.cc:697] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:37.118331 29939 consensus_queue.cc:237] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.119545 29835 raft_consensus.cc:493] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.119706 29835 raft_consensus.cc:515] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.122081 29835 leader_election.cc:290] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.123114 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e31e11048b494ebe1fd4d89488b0cf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.123875 28382 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.108752 29829 consensus_queue.cc:237] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.128295 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7be03035dc3d44f9aa5b43b49281a940" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:37.128561 28303 raft_consensus.cc:3060] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.131326 29678 ts_tablet_manager.cc:1929] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.131470 29678 log.cc:1199] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5bc3fb47136d4b88a2898af399d74a8a
I20260320 15:27:37.132326 28303 raft_consensus.cc:2468] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:37.132479 28734 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.133373 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9176e734212849aab8d99473de02cc79" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:37.133608 28583 raft_consensus.cc:3060] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.134828 28734 tablet_bootstrap.cc:654] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.136475 29835 raft_consensus.cc:493] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.136633 29835 raft_consensus.cc:515] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.136765 28583 raft_consensus.cc:2468] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:37.137264 29835 leader_election.cc:290] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.138125 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.138703 28382 leader_election.cc:343] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.139463 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16e31e11048b494ebe1fd4d89488b0cf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.142143 28736 raft_consensus.cc:359] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.142328 28736 raft_consensus.cc:385] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.142390 28736 raft_consensus.cc:740] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.143178 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.143288 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:37.143257 28903 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 100) took 3302 ms (client timeout 1000 ms). Trace:
I20260320 15:27:37.158743 29835 raft_consensus.cc:493] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.158921 29835 raft_consensus.cc:515] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.159641 29835 leader_election.cc:290] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:37.162472 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 99) took 4702 ms (client timeout 1000 ms). Trace:
W20260320 15:27:37.162676 28901 rpcz_store.cc:269] 0320 15:27:32.455669 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.325863 (+870194us) service_pool.cc:225] Handling call
0320 15:27:37.157876 (+3832013us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":235339,"reported_tablets":155}
I20260320 15:27:37.166399 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.169955 28733 tablet_bootstrap.cc:492] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.170120 28733 ts_tablet_manager.cc:1403] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.109s user 0.008s sys 0.004s
I20260320 15:27:37.170914 28733 raft_consensus.cc:359] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.171084 28733 raft_consensus.cc:385] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.171129 28733 raft_consensus.cc:740] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.171382 28733 consensus_queue.cc:260] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.171872 28733 ts_tablet_manager.cc:1434] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.175415 28733 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.176985 29835 consensus_queue.cc:1048] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:37.179520 28383 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.179682 28383 leader_election.cc:304] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.180104 29835 raft_consensus.cc:2749] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:37.180637 28383 leader_election.cc:343] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.180754 28383 leader_election.cc:304] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.181092 29835 raft_consensus.cc:2749] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.182729 29555 consensus_queue.cc:1048] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.183054 29835 raft_consensus.cc:493] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.183187 29835 raft_consensus.cc:515] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.183928 29835 leader_election.cc:290] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.184281 29678 ts_tablet_manager.cc:1950] T 5bc3fb47136d4b88a2898af399d74a8a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.184005 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.184543 29617 consensus_queue.cc:1048] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.185343 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 88) took 3253 ms. Trace:
I20260320 15:27:37.185678 29678 rpcz_store.cc:276] 0320 15:27:33.931987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.932102 (+ 115us) service_pool.cc:225] Handling call
0320 15:27:36.921122 (+2989020us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:36.921136 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.025382 (+104246us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.131184 (+105802us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.185325 (+ 54141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2929141}
I20260320 15:27:37.185521 28736 consensus_queue.cc:260] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.188267 29555 consensus_queue.cc:1048] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.188593 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.188570 28383 leader_election.cc:343] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.188812 28574 raft_consensus.cc:2468] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:37.189651 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.189797 29840 consensus_queue.cc:1048] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.190703 29676 consensus_queue.cc:1048] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.191519 29994 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:37.191702 28382 leader_election.cc:304] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.196834 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.198210 29886 raft_consensus.cc:2804] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:37.200417 29886 raft_consensus.cc:493] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:37.201280 29886 raft_consensus.cc:3060] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.201500 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 89) took 3267 ms. Trace:
I20260320 15:27:37.201732 29678 rpcz_store.cc:276] 0320 15:27:33.934503 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.934877 (+ 374us) service_pool.cc:225] Handling call
0320 15:27:37.201308 (+3266431us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:37.201329 (+ 21us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.201460 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3198189}
I20260320 15:27:37.202262 29678 tablet_replica.cc:333] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.202602 29678 raft_consensus.cc:2243] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.202908 29678 raft_consensus.cc:2272] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.203776 29678 ts_tablet_manager.cc:1916] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:37.204622 28382 leader_election.cc:343] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.204777 28382 leader_election.cc:304] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.205307 29835 raft_consensus.cc:2749] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.209419 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5bc3fb47136d4b88a2898af399d74a8a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:37.209820 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7a4b5a658cd84b04b50b54973c065889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a4b5a658cd84b04b50b54973c065889 already in progress: creating tablet
I20260320 15:27:37.214200 29640 ts_tablet_manager.cc:1929] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.214368 29640 log.cc:1199] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:37.214764 29640 ts_tablet_manager.cc:1950] T 8dc27a4968ca4203b67dd3064fb03868 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.205758 28736 ts_tablet_manager.cc:1434] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.104s user 0.001s sys 0.001s
I20260320 15:27:37.217099 28736 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:37.208127 28239 consensus_peers.cc:597] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c2d7e4d26994a17bf2d2615d6552412. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:37.146371 28903 rpcz_store.cc:269] 0320 15:27:33.840826 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.850735 (+ 9909us) service_pool.cc:225] Handling call
0320 15:27:37.143064 (+3292329us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.696335 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.696505 (+ 170us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.696513 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:34.696516 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:34.696639 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.696656 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.696661 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:34.696729 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:34.696732 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:34.696773 (+ 41us) write_op.cc:270] Start()
0320 15:27:34.696810 (+ 37us) write_op.cc:276] Timestamp: P: 1774020454696717 usec, L: 0
0320 15:27:34.696813 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.697030 (+ 217us) log.cc:844] Serialized 527 byte log entry
0320 15:27:34.702044 (+ 5014us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.702230 (+ 186us) write_op.cc:301] APPLY: starting
0320 15:27:34.702271 (+ 41us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.702336 (+ 65us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.702392 (+ 56us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.702474 (+ 82us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.702511 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:34.702523 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:34.702642 (+ 119us) log.cc:844] Serialized 23 byte log entry
0320 15:27:34.702730 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.702762 (+ 32us) write_op.cc:454] Released schema lock
0320 15:27:34.702772 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:34.702854 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.702857 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":190,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":621,"apply.run_wall_time_us":619,"num_ops":1,"prepare.queue_time_us":1076,"prepare.run_cpu_time_us":841,"prepare.run_wall_time_us":840,"raft.queue_time_us":1163,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":261,"replication_time_us":5179}]]}
I20260320 15:27:37.219585 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d1d5feef4fbc48b08f2deb15f2f277f8" } new_table_name: "default.Table_94ef14573dca40e19391aaab0e148383" modify_external_catalogs: true
I20260320 15:27:37.225292 28733 tablet_bootstrap.cc:654] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.227422 28901 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2926e23005b34db8a1d7e340b1a1e55e" } modify_external_catalogs: true
I20260320 15:27:37.228051 29886 raft_consensus.cc:515] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.230093 29886 leader_election.cc:290] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.230616 28734 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.230800 28734 ts_tablet_manager.cc:1403] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.000s
I20260320 15:27:37.229749 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:37.234170 28575 raft_consensus.cc:3060] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.237449 28575 raft_consensus.cc:2468] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:37.238793 28382 leader_election.cc:304] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.240553 29998 raft_consensus.cc:2804] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.240913 29998 raft_consensus.cc:697] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:37.241328 29998 consensus_queue.cc:237] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.243671 28734 raft_consensus.cc:359] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.243892 28734 raft_consensus.cc:385] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.243947 28734 raft_consensus.cc:740] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.246567 28734 consensus_queue.cc:260] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.249243 28734 ts_tablet_manager.cc:1434] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
I20260320 15:27:37.249608 28734 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.236702 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 87) took 3524 ms. Trace:
W20260320 15:27:37.253790 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 114) took 4379 ms (client timeout 1000 ms). Trace:
W20260320 15:27:37.254042 28889 rpcz_store.cc:269] 0320 15:27:32.874211 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.353980 (+479769us) service_pool.cc:225] Handling call
0320 15:27:34.523287 (+1169307us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.523298 (+ 11us) write_op.cc:454] Released schema lock
0320 15:27:37.253703 (+2730405us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:34.514946 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:34.515161 (+ 215us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:34.515172 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:34.515176 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:34.515308 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:34.515326 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:34.515334 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:34.515407 (+ 73us) tablet.cc:700] Row locks acquired
0320 15:27:34.515410 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:34.515452 (+ 42us) write_op.cc:270] Start()
0320 15:27:34.515502 (+ 50us) write_op.cc:276] Timestamp: P: 1774020454515409 usec, L: 0
0320 15:27:34.515505 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:34.515734 (+ 229us) log.cc:844] Serialized 527 byte log entry
0320 15:27:34.520080 (+ 4346us) op_driver.cc:464] REPLICATION: finished
0320 15:27:34.520251 (+ 171us) write_op.cc:301] APPLY: starting
0320 15:27:34.520275 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:34.520335 (+ 60us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:34.520374 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:34.520458 (+ 84us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:34.520518 (+ 60us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:34.520530 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:34.520634 (+ 104us) log.cc:844] Serialized 23 byte log entry
0320 15:27:34.520711 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:34.520740 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:34.520751 (+ 11us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":158397,"reported_tablets":193,"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":569,"apply.run_wall_time_us":567,"num_ops":1,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":910,"prepare.run_wall_time_us":911,"raft.queue_time_us":1368,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":345,"replication_time_us":4530}]]}
I20260320 15:27:37.254995 28733 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.255748 28889 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_62a50c277a2f45bd926514ab39b3b7e4" } new_table_name: "default.Table_08b880856b7f455ca758812764146420" modify_external_catalogs: true
I20260320 15:27:37.256430 28733 ts_tablet_manager.cc:1403] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.081s user 0.005s sys 0.000s
I20260320 15:27:37.256670 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:37.258704 28733 raft_consensus.cc:359] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.257718 29640 rpcz_store.cc:276] 0320 15:27:33.711942 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.712597 (+ 655us) service_pool.cc:225] Handling call
0320 15:27:36.969370 (+3256773us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:36.969381 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.126986 (+157605us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.214046 (+ 87060us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.236673 (+ 22627us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3255775}
I20260320 15:27:37.261979 28733 raft_consensus.cc:385] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.262619 28733 raft_consensus.cc:740] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.263681 28733 consensus_queue.cc:260] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.263993 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8dc27a4968ca4203b67dd3064fb03868 (table default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]) successfully deleted
I20260320 15:27:37.264746 28734 tablet_bootstrap.cc:654] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.266757 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 88) took 3554 ms. Trace:
I20260320 15:27:37.267468 28733 ts_tablet_manager.cc:1434] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260320 15:27:37.269260 29939 raft_consensus.cc:493] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:37.269428 29939 raft_consensus.cc:515] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.267833 29640 rpcz_store.cc:276] 0320 15:27:33.712062 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.712784 (+ 722us) service_pool.cc:225] Handling call
0320 15:27:37.265206 (+3552422us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:37.265422 (+ 216us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.266609 (+ 1187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3551038}
I20260320 15:27:37.269870 28733 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.270176 29939 leader_election.cc:290] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:37.270390 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ad4a30476e7146afae1e77dad078136e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ad4a30476e7146afae1e77dad078136e already in progress: creating tablet
W20260320 15:27:37.271391 28524 leader_election.cc:343] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
I20260320 15:27:37.274096 28551 tablet_service.cc:1511] Processing CreateTablet for tablet aaefdda7483e4c21b09b7c868395c6f1 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:37.279707 28733 tablet_bootstrap.cc:654] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.283020 29835 raft_consensus.cc:493] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.283234 29835 raft_consensus.cc:515] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.284140 29835 leader_election.cc:290] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:37.284348 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 95398.5us
I20260320 15:27:37.286861 28901 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2926e23005b34db8a1d7e340b1a1e55e" } modify_external_catalogs: true
W20260320 15:27:37.288125 28383 leader_election.cc:343] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.285513 29991 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:37.286499 28426 tablet_service.cc:1511] Processing CreateTablet for tablet aaefdda7483e4c21b09b7c868395c6f1 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:37.291771 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaefdda7483e4c21b09b7c868395c6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:37.286899 28734 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.292454 28734 ts_tablet_manager.cc:1403] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.000s
I20260320 15:27:37.293334 28734 raft_consensus.cc:359] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.293509 28734 raft_consensus.cc:385] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.293565 28734 raft_consensus.cc:740] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.293761 28734 consensus_queue.cc:260] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.294263 28734 ts_tablet_manager.cc:1434] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:37.294556 28734 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.298558 28734 tablet_bootstrap.cc:654] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:37.300834 28524 leader_election.cc:343] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
I20260320 15:27:37.300997 28524 leader_election.cc:304] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:37.301420 29939 raft_consensus.cc:2749] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:37.302249 28276 tablet_service.cc:1511] Processing CreateTablet for tablet aaefdda7483e4c21b09b7c868395c6f1 (DEFAULT_TABLE table=default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:37.302702 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaefdda7483e4c21b09b7c868395c6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:37.302861 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:37.308200 28383 leader_election.cc:343] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.308531 29840 consensus_queue.cc:1048] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.310286 29939 consensus_queue.cc:1048] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.311597 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.312233 28382 leader_election.cc:343] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.313732 28733 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.322118 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:37.352386 28383 leader_election.cc:343] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.352523 28383 leader_election.cc:304] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.353199 29842 raft_consensus.cc:2749] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.354043 29842 consensus_queue.cc:1048] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.355957 28736 tablet_bootstrap.cc:654] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.352185 29829 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:37.315842 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaefdda7483e4c21b09b7c868395c6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:37.366230 28733 ts_tablet_manager.cc:1403] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.096s user 0.006s sys 0.000s
I20260320 15:27:37.367131 28733 raft_consensus.cc:359] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.367304 28733 raft_consensus.cc:385] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.367348 28733 raft_consensus.cc:740] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.367589 28733 consensus_queue.cc:260] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.368041 28733 ts_tablet_manager.cc:1434] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:37.317767 29640 tablet_replica.cc:333] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.370118 29640 raft_consensus.cc:2243] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.370337 29640 raft_consensus.cc:2272] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.371086 29640 ts_tablet_manager.cc:1916] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.373507 29998 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:37.373675 29998 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.374413 29998 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.375766 29617 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:37.375924 28382 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:37.375919 29617 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.376665 29617 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.376806 29617 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:37.376924 29617 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.377516 29617 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.379832 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.380112 28444 raft_consensus.cc:2393] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
W20260320 15:27:37.381282 28524 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:37.381785 29599 consensus_queue.cc:1048] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.390285 28733 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.392194 28733 tablet_bootstrap.cc:654] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.362890 29829 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.400967 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.401176 28446 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:37.401657 28240 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.402021 29960 raft_consensus.cc:2804] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:37.402101 29960 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:37.402172 29960 raft_consensus.cc:3060] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:37.405326 29960 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.406741 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:37.406970 28446 raft_consensus.cc:3060] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:37.410005 28446 raft_consensus.cc:2468] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 2.
I20260320 15:27:37.411129 28240 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.416538 28734 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.416729 28734 ts_tablet_manager.cc:1403] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.122s user 0.008s sys 0.004s
I20260320 15:27:37.416931 29640 ts_tablet_manager.cc:1929] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.417140 29640 log.cc:1199] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/edb7b53bb7444a39b471218c6039542f
I20260320 15:27:37.417637 29640 ts_tablet_manager.cc:1950] T edb7b53bb7444a39b471218c6039542f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.417649 28734 raft_consensus.cc:359] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.417838 28734 raft_consensus.cc:385] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.417897 28734 raft_consensus.cc:740] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.418135 28734 consensus_queue.cc:260] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.418718 28734 ts_tablet_manager.cc:1434] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:37.418793 29960 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.419004 28734 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.419328 29960 raft_consensus.cc:2804] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:37.419564 29960 raft_consensus.cc:697] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:37.419895 29960 consensus_queue.cc:237] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.426193 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb65832eefe4bd4b08d95aa83c7bd7b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.430982 28734 tablet_bootstrap.cc:654] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.423039 29829 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.433547 28383 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.434222 29998 raft_consensus.cc:2749] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:37.436290 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 89) took 3722 ms. Trace:
I20260320 15:27:37.436427 29640 rpcz_store.cc:276] 0320 15:27:33.714183 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.714489 (+ 306us) service_pool.cc:225] Handling call
0320 15:27:37.271861 (+3557372us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:37.271874 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.390811 (+118937us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.416795 (+ 25984us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.436271 (+ 19476us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":768,"tablet-delete.queue_time_us":3556061}
I20260320 15:27:37.436987 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet edb7b53bb7444a39b471218c6039542f (table default.Table_1a8f81dd2a33405b819cad145fe64901 [id=7b5c27e0814148848d52b88c9dce9553]) successfully deleted
I20260320 15:27:37.437353 29640 tablet_replica.cc:333] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.427881 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.437577 29640 raft_consensus.cc:2243] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.437752 29640 raft_consensus.cc:2272] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.438326 29640 ts_tablet_manager.cc:1916] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:37.442227 28524 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:37.442441 28524 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:37.442937 29599 raft_consensus.cc:2749] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:37.451233 28239 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
W20260320 15:27:37.451751 28239 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:37.452636 28524 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:37.453261 29676 raft_consensus.cc:3060] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.454118 28733 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.455511 28733 ts_tablet_manager.cc:1403] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.065s user 0.006s sys 0.008s
I20260320 15:27:37.457736 29676 raft_consensus.cc:2749] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.460690 28733 raft_consensus.cc:359] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.462196 28733 raft_consensus.cc:385] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.462827 28733 raft_consensus.cc:740] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.464501 28733 consensus_queue.cc:260] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.483260 29643 ts_tablet_manager.cc:1929] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.488703 28733 ts_tablet_manager.cc:1434] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.033s user 0.000s sys 0.002s
I20260320 15:27:37.489198 28733 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.491057 28733 tablet_bootstrap.cc:654] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.493680 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:37.495122 29640 ts_tablet_manager.cc:1929] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.495361 29640 log.cc:1199] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b34048a722c3485490637965c60cc5dc
I20260320 15:27:37.495846 29640 ts_tablet_manager.cc:1950] T b34048a722c3485490637965c60cc5dc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.496748 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 91) took 3782 ms. Trace:
I20260320 15:27:37.496872 29643 log.cc:1199] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:37.497574 29643 ts_tablet_manager.cc:1950] T b82c47ae3786401d931bb0b486ca3570 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:37.497706 29640 rpcz_store.cc:276] 0320 15:27:33.714444 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.714573 (+ 129us) service_pool.cc:225] Handling call
0320 15:27:37.437302 (+3722729us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:37.437312 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.457874 (+ 20562us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.494974 (+ 37100us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.496727 (+ 1753us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3721283}
I20260320 15:27:37.498752 29998 raft_consensus.cc:493] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.498837 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b34048a722c3485490637965c60cc5dc (table default.Table_c11ce68814b043f0a072501aeed0ec8a [id=fcf96c12d37d4425ae3a6b2d45a0927f]) successfully deleted
I20260320 15:27:37.498905 29998 raft_consensus.cc:515] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.499349 29640 tablet_replica.cc:333] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.499576 29640 raft_consensus.cc:2243] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:37.499620 29998 leader_election.cc:290] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.500396 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.501088 28382 leader_election.cc:343] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.501366 29960 raft_consensus.cc:493] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.501578 28734 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.501725 28734 ts_tablet_manager.cc:1403] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.083s user 0.002s sys 0.003s
I20260320 15:27:37.501612 29960 raft_consensus.cc:515] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.502856 28734 raft_consensus.cc:359] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.503086 28734 raft_consensus.cc:385] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.503163 28734 raft_consensus.cc:740] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.503463 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.504413 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.504659 28583 raft_consensus.cc:2393] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
W20260320 15:27:37.505158 28383 leader_election.cc:343] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.505295 28383 leader_election.cc:304] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.505559 29998 raft_consensus.cc:2749] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.506397 29960 leader_election.cc:290] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.507434 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 93) took 3483 ms. Trace:
I20260320 15:27:37.509860 29678 ts_tablet_manager.cc:1929] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.510114 29678 log.cc:1199] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6231fe30c5b34d0bb7a233abbffeb1e8
I20260320 15:27:37.510654 29678 ts_tablet_manager.cc:1950] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.511513 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 92) took 3545 ms. Trace:
I20260320 15:27:37.511708 29678 rpcz_store.cc:276] 0320 15:27:33.992670 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.993100 (+ 430us) service_pool.cc:225] Handling call
0320 15:27:37.202199 (+3209099us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:37.202212 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.496998 (+294786us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.509697 (+ 12699us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.511495 (+ 1798us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3197916}
I20260320 15:27:37.511837 29643 rpcz_store.cc:276] 0320 15:27:34.024018 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.024082 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:37.021703 (+2997621us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:37.021714 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.467212 (+445498us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.483127 (+ 15915us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.507409 (+ 24282us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2997378}
I20260320 15:27:37.512313 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]) successfully deleted
I20260320 15:27:37.512748 29678 tablet_replica.cc:333] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.512841 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b82c47ae3786401d931bb0b486ca3570 (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
I20260320 15:27:37.513033 29678 raft_consensus.cc:2243] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.513289 29678 raft_consensus.cc:2272] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:37.513284 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 102192us
I20260320 15:27:37.514683 29678 ts_tablet_manager.cc:1916] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.515786 29643 tablet_replica.cc:333] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:37.515820 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4becc525b59d421caf79d8cd42052118" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.516100 29643 raft_consensus.cc:2243] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.516601 28240 leader_election.cc:304] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:37.516826 28240 raft_consensus.cc:2686] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:37.516988 29643 raft_consensus.cc:2272] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.517668 29643 ts_tablet_manager.cc:1916] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.521188 29640 raft_consensus.cc:2272] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.522169 29640 ts_tablet_manager.cc:1916] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.519975 28734 consensus_queue.cc:260] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.541240 29998 consensus_queue.cc:1048] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.543318 28733 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.543454 29599 raft_consensus.cc:493] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.543668 29599 raft_consensus.cc:515] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.544320 29599 leader_election.cc:290] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.544983 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.545163 29998 raft_consensus.cc:493] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.545204 28444 raft_consensus.cc:2468] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:37.545300 29998 raft_consensus.cc:515] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.545725 28524 leader_election.cc:304] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.546061 29676 raft_consensus.cc:2804] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:37.546075 29886 raft_consensus.cc:493] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.546141 29676 raft_consensus.cc:493] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:37.546212 29676 raft_consensus.cc:3060] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.546216 29886 raft_consensus.cc:515] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.546831 29886 leader_election.cc:290] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:37.547737 29886 raft_consensus.cc:493] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.547793 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.547851 29886 raft_consensus.cc:515] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.548388 29886 leader_election.cc:290] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:37.549362 28382 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.549363 29676 raft_consensus.cc:515] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.549808 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.550354 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.551225 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:37.551332 28382 leader_election.cc:343] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.551420 28444 raft_consensus.cc:3060] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:37.551887 28382 leader_election.cc:343] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.555436 28444 raft_consensus.cc:2468] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:37.556092 28524 leader_election.cc:304] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.556509 29676 leader_election.cc:290] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.556738 29599 raft_consensus.cc:2804] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.551232 29998 leader_election.cc:290] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.556866 29599 raft_consensus.cc:697] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:37.557127 29599 consensus_queue.cc:237] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.543509 28733 ts_tablet_manager.cc:1403] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.054s user 0.003s sys 0.000s
W20260320 15:27:37.558547 28382 consensus_peers.cc:597] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5f92561764b4cb9b15bb9a310a350f0. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:37.558598 28733 raft_consensus.cc:359] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.558769 28733 raft_consensus.cc:385] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.558827 28733 raft_consensus.cc:740] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.559028 28733 consensus_queue.cc:260] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.559587 28733 ts_tablet_manager.cc:1434] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.564365 28734 ts_tablet_manager.cc:1434] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.063s user 0.001s sys 0.001s
I20260320 15:27:37.564779 28734 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.565059 28733 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.566406 28734 tablet_bootstrap.cc:654] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.572809 29643 ts_tablet_manager.cc:1929] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.574420 28733 tablet_bootstrap.cc:654] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.579645 29643 log.cc:1199] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6231fe30c5b34d0bb7a233abbffeb1e8
W20260320 15:27:37.579823 30020 log.cc:927] Time spent T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.142s user 0.000s sys 0.000s
I20260320 15:27:37.580228 29643 ts_tablet_manager.cc:1950] T 6231fe30c5b34d0bb7a233abbffeb1e8 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:37.581290 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 94) took 3554 ms. Trace:
I20260320 15:27:37.583385 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.584519 29643 rpcz_store.cc:276] 0320 15:27:34.026730 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.026844 (+ 114us) service_pool.cc:225] Handling call
0320 15:27:37.513387 (+3486543us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:37.513400 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.529629 (+ 16229us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.572659 (+ 43030us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.581268 (+ 8609us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3486255}
I20260320 15:27:37.585891 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ff84dedc074dabb4e8e2d8460066e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:37.586098 29643 tablet_replica.cc:333] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:37.586092 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]) successfully deleted
I20260320 15:27:37.586299 29643 raft_consensus.cc:2243] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:37.586557 29643 pending_rounds.cc:70] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:37.586611 29643 pending_rounds.cc:77] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387790633254912 op_type: NO_OP noop_request { }
I20260320 15:27:37.586721 29643 raft_consensus.cc:2889] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:37.586812 29643 raft_consensus.cc:2272] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.587746 28734 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.587966 28734 ts_tablet_manager.cc:1403] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260320 15:27:37.588869 28734 raft_consensus.cc:359] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.589146 28734 raft_consensus.cc:385] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.589191 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.589613 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.589282 28734 raft_consensus.cc:740] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.589615 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.590118 28734 consensus_queue.cc:260] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:37.590404 28524 leader_election.cc:343] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:37.590780 28524 leader_election.cc:343] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.591005 28734 ts_tablet_manager.cc:1434] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:37.591455 28734 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:37.591737 28383 leader_election.cc:343] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.592234 28383 leader_election.cc:304] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:37.592733 28383 leader_election.cc:343] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.592829 28383 leader_election.cc:304] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.593299 29842 raft_consensus.cc:2749] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.593477 29842 consensus_queue.cc:1048] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.593874 29842 raft_consensus.cc:2749] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:37.594498 28383 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:37.594607 28524 consensus_peers.cc:597] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6231fe30c5b34d0bb7a233abbffeb1e8. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:37.594601 28383 leader_election.cc:304] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.598789 29996 raft_consensus.cc:2749] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.602593 28734 tablet_bootstrap.cc:654] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.604583 29676 raft_consensus.cc:493] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.604766 29676 raft_consensus.cc:515] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.605105 29643 ts_tablet_manager.cc:1916] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.608575 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:37.606673 30023 tablet.cc:1723] T 6ba738a76a424c18b36985a1977bcee4 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:37.613963 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:37.617575 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:37.618779 28734 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.618942 28734 ts_tablet_manager.cc:1403] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.002s
I20260320 15:27:37.619931 28734 raft_consensus.cc:359] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.620136 28734 raft_consensus.cc:385] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.620194 28734 raft_consensus.cc:740] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.618479 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:37.620484 28734 consensus_queue.cc:260] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.621196 28734 ts_tablet_manager.cc:1434] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.621563 28734 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.622486 29886 raft_consensus.cc:493] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.622639 29886 raft_consensus.cc:515] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.624071 29886 leader_election.cc:290] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.624241 28734 tablet_bootstrap.cc:654] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.626803 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:37.627642 28383 leader_election.cc:343] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.627493 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:37.630031 29886 raft_consensus.cc:493] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.630187 29886 raft_consensus.cc:515] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.630888 29886 leader_election.cc:290] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.631043 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:37.633522 29678 ts_tablet_manager.cc:1929] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:37.633652 29678 log.cc:1199] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/4becc525b59d421caf79d8cd42052118
I20260320 15:27:37.633975 29678 ts_tablet_manager.cc:1950] T 4becc525b59d421caf79d8cd42052118 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.634418 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 93) took 3620 ms. Trace:
I20260320 15:27:37.634620 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:37.634521 29678 rpcz_store.cc:276] 0320 15:27:34.013624 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.013736 (+ 112us) service_pool.cc:225] Handling call
0320 15:27:37.512698 (+3498962us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:37.512713 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.518183 (+ 5470us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.633404 (+115221us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.634395 (+ 991us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3491154}
I20260320 15:27:37.634884 29678 tablet_replica.cc:333] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.635121 29678 raft_consensus.cc:2243] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.635334 29678 raft_consensus.cc:2272] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.636148 29676 leader_election.cc:290] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.637171 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.637668 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.640324 28382 leader_election.cc:343] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.640473 28382 leader_election.cc:304] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.641233 29996 raft_consensus.cc:2749] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:37.643136 28382 leader_election.cc:343] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.644433 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.644657 28444 raft_consensus.cc:2468] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:37.645411 28524 leader_election.cc:304] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:37.647431 29640 ts_tablet_manager.cc:1929] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.649775 29640 log.cc:1199] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:37.649720 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.649619 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.650342 29640 ts_tablet_manager.cc:1950] T c752fb99ef494bda9d144d0e180e3100 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.650710 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 90) took 3936 ms. Trace:
I20260320 15:27:37.650800 29640 rpcz_store.cc:276] 0320 15:27:33.714325 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.714606 (+ 281us) service_pool.cc:225] Handling call
0320 15:27:37.499297 (+3784691us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:37.499308 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.584097 (+ 84789us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.647313 (+ 63216us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.650689 (+ 3376us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3782904}
I20260320 15:27:37.651189 28736 tablet_bootstrap.cc:492] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.651371 28736 ts_tablet_manager.cc:1403] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.434s user 0.013s sys 0.009s
I20260320 15:27:37.652208 28736 raft_consensus.cc:359] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.652410 28736 raft_consensus.cc:385] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.652485 28736 raft_consensus.cc:740] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.652729 28736 consensus_queue.cc:260] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.653395 28736 ts_tablet_manager.cc:1434] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:37.653697 28736 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:37.653923 28383 leader_election.cc:343] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.654066 28383 leader_election.cc:304] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.654458 29886 raft_consensus.cc:2749] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.651240 29676 raft_consensus.cc:2804] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:37.657086 29676 raft_consensus.cc:493] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:37.657187 29676 raft_consensus.cc:3060] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.648636 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:37.660017 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 4becc525b59d421caf79d8cd42052118 (table default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]) successfully deleted
I20260320 15:27:37.660421 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:37.661841 28905 catalog_manager.cc:5671] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
W20260320 15:27:37.655910 28524 leader_election.cc:343] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.664768 29676 raft_consensus.cc:515] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.665591 29676 leader_election.cc:290] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.666332 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:37.666543 28444 raft_consensus.cc:3060] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.666635 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:37.669363 28444 raft_consensus.cc:2468] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:37.670364 29996 consensus_queue.cc:1048] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:37.667301 28524 leader_election.cc:343] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.672360 28524 leader_election.cc:304] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:37.671686 29640 tablet_replica.cc:333] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.672884 29939 raft_consensus.cc:2804] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:37.673022 29939 raft_consensus.cc:697] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:37.673090 29640 raft_consensus.cc:2243] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.673269 29640 raft_consensus.cc:2272] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.673276 29939 consensus_queue.cc:237] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.673892 29640 ts_tablet_manager.cc:1916] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.674999 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:37.675830 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet c752fb99ef494bda9d144d0e180e3100 (table default.Table_be682052d66448438f9d7e7fc47a7ced [id=f330238fc7c246bcb43c472c5f931720]) successfully deleted
I20260320 15:27:37.677289 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:37.678685 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:37.680678 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:37.683491 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:37.684790 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:37.689409 29678 ts_tablet_manager.cc:1916] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.696549 29996 raft_consensus.cc:493] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.696717 29996 raft_consensus.cc:515] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.695394 29643 ts_tablet_manager.cc:1929] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.697510 29996 leader_election.cc:290] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.698324 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.699098 29640 ts_tablet_manager.cc:1929] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.698489 29643 log.cc:1199] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/4becc525b59d421caf79d8cd42052118
I20260320 15:27:37.699424 29640 log.cc:1199] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/5bc3fb47136d4b88a2898af399d74a8a
W20260320 15:27:37.699421 28382 leader_election.cc:343] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.699863 29640 ts_tablet_manager.cc:1950] T 5bc3fb47136d4b88a2898af399d74a8a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.700207 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 93) took 3982 ms. Trace:
I20260320 15:27:37.700312 29640 rpcz_store.cc:276] 0320 15:27:33.717575 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.717624 (+ 49us) service_pool.cc:225] Handling call
0320 15:27:37.671614 (+3953990us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:37.671627 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.695162 (+ 23535us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.698977 (+ 3815us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.700189 (+ 1212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3953712}
I20260320 15:27:37.700922 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 92) took 3984 ms. Trace:
I20260320 15:27:37.701038 29640 rpcz_store.cc:276] 0320 15:27:33.716557 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.716640 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:37.700756 (+3984116us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:37.700769 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.700901 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3981476}
I20260320 15:27:37.701493 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 94) took 3982 ms. Trace:
I20260320 15:27:37.701578 29640 rpcz_store.cc:276] 0320 15:27:33.719510 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.719572 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:37.701376 (+3981804us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:37.701386 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.701477 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3981583}
I20260320 15:27:37.701853 29640 tablet_replica.cc:333] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.702049 29640 raft_consensus.cc:2243] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.702243 29640 raft_consensus.cc:2272] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.702239 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 5bc3fb47136d4b88a2898af399d74a8a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:37.702589 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 54266dd376164673bffe829dd515b83d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54266dd376164673bffe829dd515b83d already in progress: creating tablet
W20260320 15:27:37.703351 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7a4b5a658cd84b04b50b54973c065889 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a4b5a658cd84b04b50b54973c065889 already in progress: creating tablet
I20260320 15:27:37.703817 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:37.705730 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:37.706558 28383 leader_election.cc:343] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.706715 28383 leader_election.cc:304] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.707154 29996 raft_consensus.cc:2749] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.709980 28736 tablet_bootstrap.cc:654] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.710685 29640 ts_tablet_manager.cc:1916] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.710639 29643 ts_tablet_manager.cc:1950] T 4becc525b59d421caf79d8cd42052118 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:37.718977 28734 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.719108 29640 ts_tablet_manager.cc:1929] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:37.719234 29640 log.cc:1199] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/796c907919df47bf9369e45a1248f96b
I20260320 15:27:37.719287 28734 ts_tablet_manager.cc:1403] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.098s user 0.015s sys 0.010s
I20260320 15:27:37.719578 29640 ts_tablet_manager.cc:1950] T 796c907919df47bf9369e45a1248f96b P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:37.719950 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 96) took 3977 ms. Trace:
I20260320 15:27:37.720046 29640 rpcz_store.cc:276] 0320 15:27:33.742445 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.742514 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:37.701798 (+3959284us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:37.701818 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.714860 (+ 13042us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.718979 (+ 4119us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.719926 (+ 947us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3958638}
I20260320 15:27:37.720170 28734 raft_consensus.cc:359] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.720372 28734 raft_consensus.cc:385] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.720439 28734 raft_consensus.cc:740] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.720652 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 796c907919df47bf9369e45a1248f96b (table default.Table_d1d5feef4fbc48b08f2deb15f2f277f8 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]) successfully deleted
I20260320 15:27:37.720665 28734 consensus_queue.cc:260] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.721220 28734 ts_tablet_manager.cc:1434] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:37.721446 29640 tablet_replica.cc:333] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:37.718806 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 95) took 3660 ms. Trace:
I20260320 15:27:37.724625 28734 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.722404 29643 rpcz_store.cc:276] 0320 15:27:34.058250 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.058320 (+ 70us) service_pool.cc:225] Handling call
0320 15:27:37.586039 (+3527719us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:37.586051 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.604756 (+ 18705us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x60800046dfd0. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff38d010008 00007ff38d015ee4 00007ff38d015bbd 0000000000a8b58c 00007ff37d28299e 00007ff38a48bdc3 00007ff38a583894 000000000097fcf6 000000000097fbd1 0000000000b14301
0320 15:27:37.646307 (+ 41551us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.692708 (+ 46401us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.718518 (+ 25810us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":18990336,"tablet-delete.queue_time_us":3527423}
I20260320 15:27:37.726078 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 4becc525b59d421caf79d8cd42052118 (table default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]) successfully deleted
I20260320 15:27:37.726889 28734 tablet_bootstrap.cc:654] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.728180 29643 tablet_replica.cc:333] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:37.739392 29886 consensus_queue.cc:1048] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.747104 29678 ts_tablet_manager.cc:1929] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.748180 29886 raft_consensus.cc:493] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.748394 29886 raft_consensus.cc:515] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.748312 29643 raft_consensus.cc:2243] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.749747 29886 leader_election.cc:290] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.750298 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.750043 29643 raft_consensus.cc:2272] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:37.750948 28382 leader_election.cc:343] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.752031 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:37.752534 28383 leader_election.cc:343] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.752666 28383 leader_election.cc:304] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:37.753108 29886 raft_consensus.cc:2749] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.747499 29678 log.cc:1199] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:37.754185 29678 ts_tablet_manager.cc:1950] T ea95b7a1b30e42cebcbfcd551c58f679 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.755198 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 97) took 3724 ms. Trace:
I20260320 15:27:37.756632 29678 rpcz_store.cc:276] 0320 15:27:34.030889 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.030981 (+ 92us) service_pool.cc:225] Handling call
0320 15:27:37.634829 (+3603848us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:37.634841 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.724113 (+ 89272us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.746937 (+ 22824us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.755177 (+ 8240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3603539}
I20260320 15:27:37.757375 29678 tablet_replica.cc:333] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.757885 29678 raft_consensus.cc:2243] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:37.757874 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ea95b7a1b30e42cebcbfcd551c58f679 (table default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]) successfully deleted
I20260320 15:27:37.758888 29678 raft_consensus.cc:2272] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.759985 30010 consensus_queue.cc:1048] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.760448 29643 ts_tablet_manager.cc:1916] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.762035 28736 tablet_bootstrap.cc:492] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.762217 28736 ts_tablet_manager.cc:1403] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.109s user 0.003s sys 0.004s
I20260320 15:27:37.763144 28736 raft_consensus.cc:359] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.763412 28736 raft_consensus.cc:385] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.763507 28736 raft_consensus.cc:740] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.760247 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:37.763829 28736 consensus_queue.cc:260] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.766964 28736 ts_tablet_manager.cc:1434] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:37.767372 28736 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:37.769788 28736 tablet_bootstrap.cc:654] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.768047 29678 ts_tablet_manager.cc:1916] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:37.776789 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f6240d441c624b238f7c13eb3c5fa9fc consensus state has no leader
I20260320 15:27:37.775202 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:37.779047 30010 raft_consensus.cc:493] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.779204 30010 raft_consensus.cc:515] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.779948 30010 leader_election.cc:290] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:37.780790 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:37.781021 28443 raft_consensus.cc:2393] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:37.784052 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90cd18e2ccbf48c8b804329bd3abf847" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:37.786527 28239 leader_election.cc:304] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:37.785467 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:37.786943 30010 raft_consensus.cc:3060] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.787724 29599 consensus_queue.cc:1048] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.790053 30010 raft_consensus.cc:2749] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:37.794277 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:37.800848 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:37.803052 28736 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.803330 28736 ts_tablet_manager.cc:1403] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.000s
I20260320 15:27:37.804315 28736 raft_consensus.cc:359] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.804584 28736 raft_consensus.cc:385] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.804778 28736 raft_consensus.cc:740] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.805137 28736 consensus_queue.cc:260] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.805787 28736 ts_tablet_manager.cc:1434] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.806181 28736 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:37.808250 28736 tablet_bootstrap.cc:654] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.802112 29678 ts_tablet_manager.cc:1929] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.809413 29678 log.cc:1199] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/c323b7bcb671464ea6ff584a633f10c0
I20260320 15:27:37.810151 29678 ts_tablet_manager.cc:1950] T c323b7bcb671464ea6ff584a633f10c0 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.810845 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 98) took 3779 ms. Trace:
I20260320 15:27:37.811025 29678 rpcz_store.cc:276] 0320 15:27:34.030987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031101 (+ 114us) service_pool.cc:225] Handling call
0320 15:27:37.757319 (+3726218us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:37.757331 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.792393 (+ 35062us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.801981 (+ 9588us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.810820 (+ 8839us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3725928}
I20260320 15:27:37.813558 29678 tablet_replica.cc:333] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.813966 29678 raft_consensus.cc:2243] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.814265 29678 raft_consensus.cc:2272] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.814826 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet c323b7bcb671464ea6ff584a633f10c0 (table default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]) successfully deleted
I20260320 15:27:37.816620 29678 ts_tablet_manager.cc:1916] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.816707 29599 consensus_queue.cc:1048] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.817338 28733 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.817548 28733 ts_tablet_manager.cc:1403] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.253s user 0.003s sys 0.013s
I20260320 15:27:37.818471 28733 raft_consensus.cc:359] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.818743 28733 raft_consensus.cc:385] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.818847 28733 raft_consensus.cc:740] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.819155 28733 consensus_queue.cc:260] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.819694 28733 ts_tablet_manager.cc:1434] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:37.820027 28733 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:37.821439 28734 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.821640 28734 ts_tablet_manager.cc:1403] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.097s user 0.005s sys 0.002s
I20260320 15:27:37.822676 28734 raft_consensus.cc:359] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.822947 28734 raft_consensus.cc:385] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.823055 28734 raft_consensus.cc:740] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.823372 28734 consensus_queue.cc:260] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.824277 28734 ts_tablet_manager.cc:1434] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:37.824623 28734 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.826853 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:37.833864 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:37.837976 29599 consensus_queue.cc:1048] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.830170 29842 consensus_queue.cc:1048] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.859356 28736 tablet_bootstrap.cc:492] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.860488 28733 tablet_bootstrap.cc:654] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.861485 28736 ts_tablet_manager.cc:1403] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.000s
I20260320 15:27:37.861698 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:37.864797 28736 raft_consensus.cc:359] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.873273 28736 raft_consensus.cc:385] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.871323 29678 ts_tablet_manager.cc:1929] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.874725 29678 log.cc:1199] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/89425da569794a8fa972817023bf853d
I20260320 15:27:37.875304 29678 ts_tablet_manager.cc:1950] T 89425da569794a8fa972817023bf853d P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.876963 28733 tablet_bootstrap.cc:492] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:37.880517 28733 ts_tablet_manager.cc:1403] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.061s user 0.000s sys 0.004s
I20260320 15:27:37.882525 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 104) took 3850 ms. Trace:
I20260320 15:27:37.882644 29678 rpcz_store.cc:276] 0320 15:27:34.031773 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031829 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:37.813470 (+3781641us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:37.813490 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.832988 (+ 19498us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.871189 (+ 38201us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.882485 (+ 11296us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3781400}
I20260320 15:27:37.882987 29678 tablet_replica.cc:333] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.883164 29678 raft_consensus.cc:2243] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.883311 29678 raft_consensus.cc:2272] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.883898 29678 ts_tablet_manager.cc:1916] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.873474 28736 raft_consensus.cc:740] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.887863 28736 consensus_queue.cc:260] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.885231 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:37.893999 28736 ts_tablet_manager.cc:1434] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.030s user 0.002s sys 0.001s
I20260320 15:27:37.894470 28736 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:37.896449 28734 tablet_bootstrap.cc:654] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.899036 28736 tablet_bootstrap.cc:654] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.900141 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:37.898996 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:37.913640 29840 raft_consensus.cc:493] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.913816 29840 raft_consensus.cc:515] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.914510 29840 leader_election.cc:290] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.914614 28736 tablet_bootstrap.cc:492] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:37.914685 29840 consensus_queue.cc:1048] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.914760 28736 ts_tablet_manager.cc:1403] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.002s
I20260320 15:27:37.915700 28736 raft_consensus.cc:359] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.915951 28736 raft_consensus.cc:385] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.916054 28736 raft_consensus.cc:740] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.916281 28736 consensus_queue.cc:260] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.916990 28736 ts_tablet_manager.cc:1434] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.917308 28736 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:37.917790 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 89425da569794a8fa972817023bf853d (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:37.919073 28736 tablet_bootstrap.cc:654] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.924377 29643 ts_tablet_manager.cc:1929] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:37.924749 29643 log.cc:1199] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:37.925716 29643 ts_tablet_manager.cc:1950] T 16e31e11048b494ebe1fd4d89488b0cf P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:37.926296 30010 consensus_queue.cc:1048] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.915670 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_313ec95c683c4f99adb04f3968de7963 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 4 got 3
I20260320 15:27:37.929518 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:37.930281 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:37.930788 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:37.931713 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:37.933238 29842 consensus_queue.cc:1048] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.921067 29678 ts_tablet_manager.cc:1929] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:37.934335 29678 log.cc:1199] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f70530cd63a0420fb51527759c951e31
I20260320 15:27:37.935143 29678 ts_tablet_manager.cc:1950] T f70530cd63a0420fb51527759c951e31 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:37.942891 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 96) took 3912 ms. Trace:
I20260320 15:27:37.943704 29678 rpcz_store.cc:276] 0320 15:27:34.030746 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.032199 (+ 1453us) service_pool.cc:225] Handling call
0320 15:27:37.882935 (+3850736us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:37.882947 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.903579 (+ 20632us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.920958 (+ 17379us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.942862 (+ 21904us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3850529}
I20260320 15:27:37.944659 29678 tablet_replica.cc:333] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:37.952281 29678 raft_consensus.cc:2243] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.952666 29678 raft_consensus.cc:2272] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:37.894291 29617 raft_consensus.cc:493] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:37.958619 29617 raft_consensus.cc:515] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:37.959287 29617 leader_election.cc:290] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:37.905930 29676 consensus_queue.cc:1048] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.917294 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:37.929639 28524 consensus_peers.cc:597] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:37.960299 28736 tablet_bootstrap.cc:492] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
W20260320 15:27:37.961347 28382 leader_election.cc:343] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:37.961699 28736 ts_tablet_manager.cc:1403] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.002s
I20260320 15:27:37.966749 30010 consensus_queue.cc:1048] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.969308 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:37.969831 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet f70530cd63a0420fb51527759c951e31 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:37.970868 28295 raft_consensus.cc:3060] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:37.971805 29678 ts_tablet_manager.cc:1916] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:37.974736 28736 raft_consensus.cc:359] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.974965 28736 raft_consensus.cc:385] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.975054 28736 raft_consensus.cc:740] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.975340 28736 consensus_queue.cc:260] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.975982 28736 ts_tablet_manager.cc:1434] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20260320 15:27:37.976369 28736 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:37.960376 30056 tablet.cc:1723] T db81df6d3e9d49efb94c4f24de9db429 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:37.986781 28734 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:37.965999 29640 raft_consensus.cc:2243] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:37.987073 28734 ts_tablet_manager.cc:1403] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.163s user 0.004s sys 0.002s
I20260320 15:27:37.987138 29640 raft_consensus.cc:2272] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:37.966233 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:37.988101 28734 raft_consensus.cc:359] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.988288 28734 raft_consensus.cc:385] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.988348 28734 raft_consensus.cc:740] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.988389 29676 consensus_queue.cc:1048] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:37.988551 28734 consensus_queue.cc:260] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:37.989195 28734 ts_tablet_manager.cc:1434] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:37.989488 28734 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:37.990010 28736 tablet_bootstrap.cc:654] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:37.990963 29640 ts_tablet_manager.cc:1916] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:37.996963 28524 consensus_peers.cc:597] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6231fe30c5b34d0bb7a233abbffeb1e8. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:37.961438 28733 raft_consensus.cc:359] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.998030 28733 raft_consensus.cc:385] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:37.998183 28733 raft_consensus.cc:740] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:37.998970 28733 consensus_queue.cc:260] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:37.999909 29678 ts_tablet_manager.cc:1929] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.000038 29678 log.cc:1199] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:38.000447 29678 ts_tablet_manager.cc:1950] T 66a181681cd84b7fb09c3742f4f2222f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.002727 28733 ts_tablet_manager.cc:1434] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.042s user 0.000s sys 0.002s
I20260320 15:27:38.003122 28733 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.003401 28734 tablet_bootstrap.cc:654] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.004904 28736 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.001266 29640 ts_tablet_manager.cc:1929] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.005059 28736 ts_tablet_manager.cc:1403] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.001s
I20260320 15:27:38.005131 29640 log.cc:1199] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:38.005648 29640 ts_tablet_manager.cc:1950] T 931f6f88a1d94a40b69f07216b8313c8 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.005884 28736 raft_consensus.cc:359] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.006076 28736 raft_consensus.cc:385] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.006127 28736 raft_consensus.cc:740] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.006153 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 97) took 4262 ms. Trace:
I20260320 15:27:38.006333 28736 consensus_queue.cc:260] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.006332 29640 rpcz_store.cc:276] 0320 15:27:33.743373 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.743418 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:37.721387 (+3977969us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:37.721400 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.994046 (+272646us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.001160 (+ 7114us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.006038 (+ 4878us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":244183,"tablet-delete.queue_time_us":3977572}
I20260320 15:27:38.006613 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 103) took 3974 ms. Trace:
I20260320 15:27:38.006757 29678 rpcz_store.cc:276] 0320 15:27:34.031650 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.032440 (+ 790us) service_pool.cc:225] Handling call
0320 15:27:37.944599 (+3912159us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:37.944613 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.979764 (+ 35151us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.999792 (+ 20028us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.006593 (+ 6801us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8501,"tablet-delete.queue_time_us":3906426}
I20260320 15:27:38.007035 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 931f6f88a1d94a40b69f07216b8313c8 (table default.Table_e25b7a014de24f8c8f22a3584fd0d367 [id=19af05ab6c6544b99f6b348f2c52b62b]) successfully deleted
I20260320 15:27:38.007180 29678 tablet_replica.cc:333] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.007367 29678 raft_consensus.cc:2243] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.007433 29640 tablet_replica.cc:333] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.007570 29678 raft_consensus.cc:2272] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.007754 29640 raft_consensus.cc:2243] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.008098 29640 raft_consensus.cc:2272] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.008188 29678 ts_tablet_manager.cc:1916] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.008740 29640 ts_tablet_manager.cc:1916] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.006922 28736 ts_tablet_manager.cc:1434] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.013751 28736 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.014511 30010 raft_consensus.cc:493] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.014639 30010 raft_consensus.cc:515] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
W20260320 15:27:38.014775 28383 leader_election.cc:343] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.014950 28383 leader_election.cc:304] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.015290 30010 leader_election.cc:290] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.015504 29840 raft_consensus.cc:2749] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.016149 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "796c907919df47bf9369e45a1248f96b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.016359 28441 raft_consensus.cc:2468] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
W20260320 15:27:38.016435 28383 consensus_peers.cc:597] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.016794 28240 leader_election.cc:304] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.016832 29840 consensus_queue.cc:1048] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.017167 30010 raft_consensus.cc:2804] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.017246 30010 raft_consensus.cc:493] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.017300 30010 raft_consensus.cc:3060] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:38.017568 28524 leader_election.cc:343] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:38.017688 28524 leader_election.cc:304] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.018051 29617 raft_consensus.cc:2749] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.020072 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 66a181681cd84b7fb09c3742f4f2222f (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:38.020818 30010 raft_consensus.cc:515] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.021808 30010 leader_election.cc:290] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.022535 28736 tablet_bootstrap.cc:654] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.023803 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "796c907919df47bf9369e45a1248f96b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.023855 30010 consensus_queue.cc:1048] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.024005 28441 raft_consensus.cc:3060] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:38.025624 28239 leader_election.cc:343] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b
I20260320 15:27:38.027834 30055 tablet.cc:1723] T db81df6d3e9d49efb94c4f24de9db429 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:38.037176 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 96) took 3978 ms. Trace:
I20260320 15:27:38.037302 29643 rpcz_store.cc:276] 0320 15:27:34.058478 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.058570 (+ 92us) service_pool.cc:225] Handling call
0320 15:27:37.727151 (+3668581us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:37.727439 (+ 288us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:37.829598 (+102159us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:37.924014 (+ 94416us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.037144 (+113130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3668001}
I20260320 15:27:38.038453 29643 tablet_replica.cc:333] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:38.038883 28441 raft_consensus.cc:2468] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.038676 29643 raft_consensus.cc:2243] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.039391 28240 leader_election.cc:304] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.039485 29643 raft_consensus.cc:2272] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.039882 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 16e31e11048b494ebe1fd4d89488b0cf (table default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]) successfully deleted
I20260320 15:27:38.041642 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:38.041710 29618 raft_consensus.cc:2804] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.041831 29618 raft_consensus.cc:697] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.042071 29618 consensus_queue.cc:237] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.042209 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:38.042521 28905 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:38.051045 28733 tablet_bootstrap.cc:654] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.064589 29678 ts_tablet_manager.cc:1929] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.064742 29678 log.cc:1199] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/bcec6175c1ca49f5b682a29d5c1e235a
I20260320 15:27:38.065120 29678 ts_tablet_manager.cc:1950] T bcec6175c1ca49f5b682a29d5c1e235a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.065483 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 94) took 4039 ms. Trace:
I20260320 15:27:38.065577 29678 rpcz_store.cc:276] 0320 15:27:34.026497 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.026620 (+ 123us) service_pool.cc:225] Handling call
0320 15:27:38.007131 (+3980511us) ts_tablet_manager.cc:1151] Deleting tablet bcec6175c1ca49f5b682a29d5c1e235a
0320 15:27:38.007142 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.013253 (+ 6111us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.064459 (+ 51206us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.065465 (+ 1006us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3968546}
I20260320 15:27:38.066001 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 99) took 4034 ms. Trace:
I20260320 15:27:38.066078 29678 rpcz_store.cc:276] 0320 15:27:34.031084 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031413 (+ 329us) service_pool.cc:225] Handling call
0320 15:27:38.065867 (+4034454us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:38.065877 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.065985 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4025872}
I20260320 15:27:38.066409 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 100) took 4035 ms. Trace:
I20260320 15:27:38.066475 29678 rpcz_store.cc:276] 0320 15:27:34.031272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031475 (+ 203us) service_pool.cc:225] Handling call
0320 15:27:38.066305 (+4034830us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:38.066315 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.066394 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4026127}
I20260320 15:27:38.066702 29678 tablet_replica.cc:333] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.066898 29678 raft_consensus.cc:2243] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.067072 29678 raft_consensus.cc:2272] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.067855 29678 ts_tablet_manager.cc:1916] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.041229 29643 ts_tablet_manager.cc:1916] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.092744 29840 raft_consensus.cc:493] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.092942 29840 raft_consensus.cc:515] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.093699 29840 leader_election.cc:290] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:38.094698 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:38.095357 28382 leader_election.cc:343] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.095675 29939 consensus_queue.cc:1048] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.096894 28733 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.097070 28733 ts_tablet_manager.cc:1403] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.094s user 0.003s sys 0.001s
I20260320 15:27:38.098958 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet bcec6175c1ca49f5b682a29d5c1e235a (table default.Table_d1cf5412b72c4a58931061b1a1aba1e6 [id=fc92586ccab74ae3960a1eacc8cbac42]) successfully deleted
W20260320 15:27:38.101390 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ac4c7dc6859048ce956927201df44a86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac4c7dc6859048ce956927201df44a86 already in progress: creating tablet
W20260320 15:27:38.103736 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fa5618796f1a4ff3ab4c1209dad0aede because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa5618796f1a4ff3ab4c1209dad0aede already in progress: creating tablet
I20260320 15:27:38.107813 29678 ts_tablet_manager.cc:1929] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.108053 29678 log.cc:1199] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:38.108600 29678 ts_tablet_manager.cc:1950] T 9a072859f39c4c28ae65b0e9afa79d66 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.110741 29840 raft_consensus.cc:493] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.110896 29840 raft_consensus.cc:515] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.111866 29840 leader_election.cc:290] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.111980 30023 tablet.cc:1723] T db81df6d3e9d49efb94c4f24de9db429 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:38.116108 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:38.116822 28382 leader_election.cc:343] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.119861 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 101) took 4088 ms. Trace:
I20260320 15:27:38.120095 29678 rpcz_store.cc:276] 0320 15:27:34.031386 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031515 (+ 129us) service_pool.cc:225] Handling call
0320 15:27:38.066658 (+4035143us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:38.066666 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.071312 (+ 4646us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.107663 (+ 36351us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.119831 (+ 12168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4026291}
I20260320 15:27:38.120775 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 102) took 4089 ms. Trace:
I20260320 15:27:38.120937 29678 rpcz_store.cc:276] 0320 15:27:34.031483 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.031560 (+ 77us) service_pool.cc:225] Handling call
0320 15:27:38.120600 (+4089040us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:38.120615 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.120754 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4079652}
I20260320 15:27:38.121362 29678 tablet_replica.cc:333] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.121697 29678 raft_consensus.cc:2243] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.121994 29678 raft_consensus.cc:2272] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.123064 29678 ts_tablet_manager.cc:1916] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.110193 28733 raft_consensus.cc:359] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.132495 28733 raft_consensus.cc:385] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.132577 28733 raft_consensus.cc:740] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.132972 28733 consensus_queue.cc:260] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.134284 28733 ts_tablet_manager.cc:1434] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.037s user 0.002s sys 0.001s
I20260320 15:27:38.135107 28733 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.141391 28734 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.141584 28734 ts_tablet_manager.cc:1403] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.152s user 0.016s sys 0.035s
I20260320 15:27:38.141795 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.142096 28736 tablet_bootstrap.cc:492] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.142236 28736 ts_tablet_manager.cc:1403] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.129s user 0.003s sys 0.003s
I20260320 15:27:38.142442 28734 raft_consensus.cc:359] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.142627 28734 raft_consensus.cc:385] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.142684 28734 raft_consensus.cc:740] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.142926 28734 consensus_queue.cc:260] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.143146 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.143333 28736 raft_consensus.cc:359] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.143585 28736 raft_consensus.cc:385] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.143627 28734 ts_tablet_manager.cc:1434] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.143661 28736 raft_consensus.cc:740] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.143903 28734 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.143944 28736 consensus_queue.cc:260] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.144726 28736 ts_tablet_manager.cc:1434] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260320 15:27:38.144938 28524 consensus_peers.cc:597] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f70530cd63a0420fb51527759c951e31. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.145079 28736 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.145697 28734 tablet_bootstrap.cc:654] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.147002 28736 tablet_bootstrap.cc:654] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.148603 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9a072859f39c4c28ae65b0e9afa79d66 (table default.Table_874d6a667bff4c3d815fcff90cae13f5 [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
W20260320 15:27:38.148968 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
W20260320 15:27:38.149375 28383 leader_election.cc:343] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.149520 28383 leader_election.cc:304] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.149885 29840 raft_consensus.cc:2749] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:38.150872 28383 leader_election.cc:343] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.150990 28383 leader_election.cc:304] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.151516 29840 raft_consensus.cc:2749] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.155426 29599 raft_consensus.cc:493] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.155653 29599 raft_consensus.cc:515] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.156342 29599 leader_election.cc:290] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.157305 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.157582 28441 raft_consensus.cc:2468] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.158035 28524 leader_election.cc:304] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.158434 29599 raft_consensus.cc:2804] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.158547 29599 raft_consensus.cc:493] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.158623 29599 raft_consensus.cc:3060] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.162448 29599 raft_consensus.cc:515] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.163856 29599 leader_election.cc:290] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.163933 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.164177 28441 raft_consensus.cc:3060] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.167106 29678 ts_tablet_manager.cc:1929] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.167258 29678 log.cc:1199] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:38.167742 29678 ts_tablet_manager.cc:1950] T f5f92561764b4cb9b15bb9a310a350f0 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.167932 29599 raft_consensus.cc:493] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.168102 29599 raft_consensus.cc:515] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.168679 28441 raft_consensus.cc:2468] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.169988 28524 leader_election.cc:304] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.170271 29617 raft_consensus.cc:2804] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.170372 29617 raft_consensus.cc:697] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.170614 29617 consensus_queue.cc:237] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.175097 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:38.175793 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:38.176029 29599 leader_election.cc:290] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.175958 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.176211 28441 raft_consensus.cc:2468] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.176352 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:38.176612 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:38.176726 28524 leader_election.cc:304] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.177057 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:38.177083 29599 raft_consensus.cc:2804] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.177153 29599 raft_consensus.cc:493] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.177234 29599 raft_consensus.cc:3060] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.177661 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:38.177973 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:38.178575 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:38.179037 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:38.180560 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:38.182781 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:38.183142 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:38.182358 29599 raft_consensus.cc:515] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.187839 29939 raft_consensus.cc:493] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.188069 29939 raft_consensus.cc:515] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.188714 29939 leader_election.cc:290] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.190151 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:38.190793 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:38.191386 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:38.192502 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:38.193195 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:38.194169 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:38.197906 29599 leader_election.cc:290] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.201833 28733 tablet_bootstrap.cc:654] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.204391 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 105) took 4141 ms. Trace:
I20260320 15:27:38.205214 29678 rpcz_store.cc:276] 0320 15:27:34.063327 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.063378 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:38.121305 (+4057927us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:38.121317 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.132197 (+ 10880us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.166950 (+ 34753us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.204364 (+ 37414us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4054536}
I20260320 15:27:38.207957 29678 tablet_replica.cc:333] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.208245 29678 raft_consensus.cc:2243] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.208654 29678 raft_consensus.cc:2272] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.209321 29678 ts_tablet_manager.cc:1916] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.210871 28736 tablet_bootstrap.cc:492] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.211715 28736 ts_tablet_manager.cc:1403] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.006s
I20260320 15:27:38.212598 28736 raft_consensus.cc:359] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.212900 28736 raft_consensus.cc:385] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.213001 28736 raft_consensus.cc:740] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.213285 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.219017 28297 raft_consensus.cc:2468] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.213501 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef09e4afdd1f4fce9caec4dcafa2c50c" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:38.219707 28303 raft_consensus.cc:3060] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.222993 28303 raft_consensus.cc:2468] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.213680 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.223744 28304 raft_consensus.cc:2468] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.224970 28736 consensus_queue.cc:260] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.225819 28736 ts_tablet_manager.cc:1434] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260320 15:27:38.214460 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:38.226193 28302 raft_consensus.cc:3060] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.214929 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.227110 28295 raft_consensus.cc:2468] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.227598 28736 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.229483 28736 tablet_bootstrap.cc:654] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.191694 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.237110 29643 ts_tablet_manager.cc:1929] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.237376 28441 raft_consensus.cc:2468] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.237434 29643 log.cc:1199] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/c323b7bcb671464ea6ff584a633f10c0
I20260320 15:27:38.237506 29840 raft_consensus.cc:493] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.237637 29840 raft_consensus.cc:515] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.238955 28524 leader_election.cc:304] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.238917 29643 ts_tablet_manager.cc:1950] T c323b7bcb671464ea6ff584a633f10c0 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:38.239564 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0a31d385b81456fb46ea52083874fcf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.239732 28444 raft_consensus.cc:3060] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.240041 29599 raft_consensus.cc:2804] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.240150 29599 raft_consensus.cc:493] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.240294 29599 raft_consensus.cc:3060] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.241464 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:38.241770 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:38.230477 28302 raft_consensus.cc:2468] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.249119 29640 ts_tablet_manager.cc:1929] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.249765 29640 log.cc:1199] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:38.250360 30010 raft_consensus.cc:493] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.250551 30010 raft_consensus.cc:515] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.251325 30010 leader_election.cc:290] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.252100 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.253333 29640 ts_tablet_manager.cc:1950] T 5438fdc6e6944e79ac8aecd0ef774467 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.238373 30018 consensus_queue.cc:1048] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:38.214954 28239 leader_election.cc:343] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b
I20260320 15:27:38.254174 30010 consensus_queue.cc:1048] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.255278 29840 leader_election.cc:290] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.255967 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.256270 28524 leader_election.cc:304] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.256536 29939 raft_consensus.cc:2804] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.256654 29939 raft_consensus.cc:697] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.256780 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea6ff59d2ffe4d579e6db2375eb11414" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.256889 29939 consensus_queue.cc:237] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.257041 28574 raft_consensus.cc:2393] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
W20260320 15:27:38.257161 28382 leader_election.cc:343] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:38.260272 28383 consensus_peers.cc:597] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89425da569794a8fa972817023bf853d. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.260783 28239 leader_election.cc:304] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.263372 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.264520 30018 raft_consensus.cc:3060] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.264588 29842 consensus_queue.cc:1048] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.269604 28736 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.269795 28736 ts_tablet_manager.cc:1403] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.000s
I20260320 15:27:38.270663 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 95) took 4528 ms. Trace:
I20260320 15:27:38.270774 29998 raft_consensus.cc:493] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.270947 29998 raft_consensus.cc:515] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.270843 29640 rpcz_store.cc:276] 0320 15:27:33.742260 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.742701 (+ 441us) service_pool.cc:225] Handling call
0320 15:27:38.007386 (+4264685us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:38.007397 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.128873 (+121476us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.207589 (+ 78716us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.270626 (+ 63037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4263433}
I20260320 15:27:38.271663 29998 leader_election.cc:290] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.271895 29640 tablet_replica.cc:333] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.271886 28736 raft_consensus.cc:359] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.272248 29640 raft_consensus.cc:2243] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.272276 28736 raft_consensus.cc:385] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.272428 29640 raft_consensus.cc:2272] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.272411 28736 raft_consensus.cc:740] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.272899 28736 consensus_queue.cc:260] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.273200 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.273180 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet f5f92561764b4cb9b15bb9a310a350f0 (table default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]) successfully deleted
W20260320 15:27:38.273869 28382 leader_election.cc:343] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.274444 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 5438fdc6e6944e79ac8aecd0ef774467 (table default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]) successfully deleted
I20260320 15:27:38.257970 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_313ec95c683c4f99adb04f3968de7963 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 4 got 3
I20260320 15:27:38.275879 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:38.277225 28736 ts_tablet_manager.cc:1434] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260320 15:27:38.271677 28734 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.280282 28734 ts_tablet_manager.cc:1403] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.136s user 0.047s sys 0.031s
I20260320 15:27:38.280607 28444 raft_consensus.cc:2468] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.281121 28734 raft_consensus.cc:359] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.281337 28734 raft_consensus.cc:385] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.281392 28734 raft_consensus.cc:740] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.281562 28736 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.277232 30018 raft_consensus.cc:2749] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.287415 29599 raft_consensus.cc:515] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.291042 28734 consensus_queue.cc:260] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.292841 28736 tablet_bootstrap.cc:654] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.295444 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:38.296244 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:38.296598 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:38.296989 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:38.297222 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:38.297434 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:38.297533 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.297724 28444 raft_consensus.cc:3060] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.297935 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:38.298040 28734 ts_tablet_manager.cc:1434] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.018s user 0.000s sys 0.002s
I20260320 15:27:38.298252 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:38.298463 28734 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.298588 28904 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:38.313825 28736 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.314182 28736 ts_tablet_manager.cc:1403] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.011s
I20260320 15:27:38.318147 28444 raft_consensus.cc:2468] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.318388 28736 raft_consensus.cc:359] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.319440 28736 raft_consensus.cc:385] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.319649 28736 raft_consensus.cc:740] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.320237 28736 consensus_queue.cc:260] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.321175 28524 leader_election.cc:304] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.321516 29939 raft_consensus.cc:2804] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.321692 29939 raft_consensus.cc:697] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.321945 29939 consensus_queue.cc:237] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
W20260320 15:27:38.322108 28383 consensus_peers.cc:597] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a072859f39c4c28ae65b0e9afa79d66. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.323408 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 97) took 4246 ms. Trace:
I20260320 15:27:38.323704 28734 tablet_bootstrap.cc:654] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.323580 29643 rpcz_store.cc:276] 0320 15:27:34.076766 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.076839 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:38.038392 (+3961553us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:38.038404 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.168053 (+129649us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.236488 (+ 68435us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.323381 (+ 86893us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3961257}
I20260320 15:27:38.324469 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet c323b7bcb671464ea6ff584a633f10c0 (table default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]) successfully deleted
I20260320 15:27:38.325822 29643 tablet_replica.cc:333] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:38.326167 29643 raft_consensus.cc:2243] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.330022 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa54115d4df74bec85970057b5c3dd51" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:38.330236 28303 raft_consensus.cc:3060] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.331650 29640 ts_tablet_manager.cc:1916] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.327443 28734 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
W20260320 15:27:38.329303 28383 leader_election.cc:343] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.333981 28383 leader_election.cc:304] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.334316 28303 raft_consensus.cc:2468] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.335067 29840 raft_consensus.cc:2749] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.335609 28734 ts_tablet_manager.cc:1403] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.001s
I20260320 15:27:38.335624 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.336834 28734 raft_consensus.cc:359] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.337076 28734 raft_consensus.cc:385] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.337147 28734 raft_consensus.cc:740] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.337451 28734 consensus_queue.cc:260] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.338141 28734 ts_tablet_manager.cc:1434] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:38.338570 28734 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.339978 29840 raft_consensus.cc:493] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.340190 29840 raft_consensus.cc:515] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.341145 29840 leader_election.cc:290] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.342044 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:38.347021 28382 leader_election.cc:343] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.347024 28736 ts_tablet_manager.cc:1434] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.033s user 0.004s sys 0.000s
I20260320 15:27:38.350847 28734 tablet_bootstrap.cc:654] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.295311 29599 leader_election.cc:290] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.367148 28733 tablet_bootstrap.cc:492] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.367332 28733 ts_tablet_manager.cc:1403] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.232s user 0.005s sys 0.001s
I20260320 15:27:38.368361 28733 raft_consensus.cc:359] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.368610 28733 raft_consensus.cc:385] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.368665 28733 raft_consensus.cc:740] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.368808 29678 ts_tablet_manager.cc:1929] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.368873 28733 consensus_queue.cc:260] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.369391 28733 ts_tablet_manager.cc:1434] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:38.369766 28733 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.370371 28736 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.368986 29678 log.cc:1199] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:38.371064 29678 ts_tablet_manager.cc:1950] T bb7a910bd2904a40ae63608377206c7e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.372109 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 95) took 4343 ms. Trace:
I20260320 15:27:38.372198 29643 raft_consensus.cc:2272] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.372956 29643 ts_tablet_manager.cc:1916] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.372592 28736 tablet_bootstrap.cc:654] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.377894 29678 rpcz_store.cc:276] 0320 15:27:34.028388 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.029608 (+ 1220us) service_pool.cc:225] Handling call
0320 15:27:38.207899 (+4178291us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:38.207911 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.252526 (+ 44615us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.366683 (+114157us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.372083 (+ 5400us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4140720}
I20260320 15:27:38.378661 29678 tablet_replica.cc:333] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.379089 28733 tablet_bootstrap.cc:654] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.383770 29678 raft_consensus.cc:2243] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.385362 29678 raft_consensus.cc:2272] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.388644 29678 ts_tablet_manager.cc:1916] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:38.398455 28383 leader_election.cc:343] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.398634 28383 leader_election.cc:304] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.399190 29842 raft_consensus.cc:2749] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.389596 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.401585 28736 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.403955 28736 ts_tablet_manager.cc:1403] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.006s
I20260320 15:27:38.404412 29842 consensus_queue.cc:1048] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.406340 28736 raft_consensus.cc:359] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.406822 28736 raft_consensus.cc:385] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.407027 28736 raft_consensus.cc:740] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.408026 28736 consensus_queue.cc:260] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.413111 28733 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.413487 28733 ts_tablet_manager.cc:1403] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.044s user 0.006s sys 0.008s
I20260320 15:27:38.414142 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:38.414446 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:38.414633 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:38.414824 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:38.414963 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:38.415376 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:38.415753 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:38.415798 28733 raft_consensus.cc:359] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.416180 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:38.416209 28733 raft_consensus.cc:385] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.416465 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:38.416399 28733 raft_consensus.cc:740] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.410903 28736 ts_tablet_manager.cc:1434] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260320 15:27:38.417282 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:38.417812 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:38.418036 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:38.419016 28733 consensus_queue.cc:260] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.420097 28736 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:38.425122 28383 leader_election.cc:343] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.425307 28383 leader_election.cc:304] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.424494 28733 ts_tablet_manager.cc:1434] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260320 15:27:38.426245 29886 raft_consensus.cc:2749] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.427094 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet bb7a910bd2904a40ae63608377206c7e (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
I20260320 15:27:38.429333 28733 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.430718 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:38.431087 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:38.431277 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:38.431466 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:38.431986 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:38.433755 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:38.434301 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:38.434830 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:38.435021 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:38.435830 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:38.436465 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:38.436930 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6ba738a76a424c18b36985a1977bcee4 (table default.Table_313ec95c683c4f99adb04f3968de7963 [id=56aecaca2e3a4c868f419fa30e404f7d]). Expected version 4 got 3
I20260320 15:27:38.438155 28734 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.438381 28734 ts_tablet_manager.cc:1403] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.100s user 0.007s sys 0.001s
I20260320 15:27:38.439275 28734 raft_consensus.cc:359] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.439280 28736 tablet_bootstrap.cc:654] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.439576 28734 raft_consensus.cc:385] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.439675 28734 raft_consensus.cc:740] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.439930 28734 consensus_queue.cc:260] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.412405 29678 ts_tablet_manager.cc:1929] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.440363 29678 log.cc:1199] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:38.440682 28734 ts_tablet_manager.cc:1434] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.440809 29678 ts_tablet_manager.cc:1950] T a4bfb87d5d5741adb0ad2f2a88f4867f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.441776 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 106) took 4377 ms. Trace:
I20260320 15:27:38.441885 29678 rpcz_store.cc:276] 0320 15:27:34.066909 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.066957 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:38.378598 (+4311641us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:38.378616 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.407470 (+ 28854us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.412234 (+ 4764us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.441750 (+ 29516us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4309171}
I20260320 15:27:38.442238 29678 tablet_replica.cc:333] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.442437 29678 raft_consensus.cc:2243] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.442857 29678 raft_consensus.cc:2272] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.443470 29678 ts_tablet_manager.cc:1916] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.450076 28733 tablet_bootstrap.cc:654] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.457737 28734 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.458494 28736 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.458877 28736 ts_tablet_manager.cc:1403] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.005s
I20260320 15:27:38.459933 28736 raft_consensus.cc:359] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.460162 28736 raft_consensus.cc:385] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.460232 28736 raft_consensus.cc:740] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.460566 28736 consensus_queue.cc:260] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.461097 28736 ts_tablet_manager.cc:1434] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:38.461403 28736 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:38.464030 28383 consensus_peers.cc:597] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5f92561764b4cb9b15bb9a310a350f0. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.464804 28304 raft_consensus.cc:3060] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.462399 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet a4bfb87d5d5741adb0ad2f2a88f4867f (table default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]) successfully deleted
I20260320 15:27:38.474309 29939 raft_consensus.cc:493] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.474516 29939 raft_consensus.cc:515] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.474961 28736 tablet_bootstrap.cc:654] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.475310 29939 leader_election.cc:290] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.477409 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.477675 28444 raft_consensus.cc:2468] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.479110 28524 leader_election.cc:304] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.479779 29939 raft_consensus.cc:2804] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.479859 29939 raft_consensus.cc:493] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.479915 29939 raft_consensus.cc:3060] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.483137 29939 raft_consensus.cc:515] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.484474 28733 tablet_bootstrap.cc:492] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.484638 28733 ts_tablet_manager.cc:1403] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.000s
I20260320 15:27:38.485658 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.485873 28444 raft_consensus.cc:3060] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.485749 28906 catalog_manager.cc:5671] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:38.485723 28733 raft_consensus.cc:359] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.486158 28733 raft_consensus.cc:385] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.486183 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:38.486219 28733 raft_consensus.cc:740] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.486442 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:38.486502 28733 consensus_queue.cc:260] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.487028 28733 ts_tablet_manager.cc:1434] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:38.488814 28733 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.490114 28444 raft_consensus.cc:2468] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.490974 28733 tablet_bootstrap.cc:654] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.491201 28524 leader_election.cc:304] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.491549 29599 raft_consensus.cc:2804] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.491684 29599 raft_consensus.cc:697] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.491953 29599 consensus_queue.cc:237] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.493582 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.494086 29676 consensus_queue.cc:1048] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:38.494941 28524 leader_election.cc:343] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.498025 29939 leader_election.cc:290] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.499989 29842 raft_consensus.cc:493] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.500205 29842 raft_consensus.cc:515] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.500938 29842 leader_election.cc:290] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.501529 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.502148 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:38.502239 28382 leader_election.cc:343] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.504995 28734 tablet_bootstrap.cc:654] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.506770 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:38.509159 28524 leader_election.cc:343] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.511682 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:38.518893 28733 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.519083 28733 ts_tablet_manager.cc:1403] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.006s
I20260320 15:27:38.520895 28733 raft_consensus.cc:359] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.521104 28733 raft_consensus.cc:385] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.521157 28733 raft_consensus.cc:740] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.521413 28733 consensus_queue.cc:260] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.521821 30010 raft_consensus.cc:493] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.522002 28733 ts_tablet_manager.cc:1434] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:38.521982 30010 raft_consensus.cc:515] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.522089 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:38.522161 28736 tablet_bootstrap.cc:492] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.522522 28736 ts_tablet_manager.cc:1403] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.061s user 0.008s sys 0.003s
I20260320 15:27:38.522671 30010 leader_election.cc:290] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:38.523767 28524 consensus_peers.cc:597] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcec6175c1ca49f5b682a29d5c1e235a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.523902 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:38.524443 28239 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.525017 28736 raft_consensus.cc:359] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:38.508517 28383 leader_election.cc:343] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.525437 28736 raft_consensus.cc:385] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.525665 28383 leader_election.cc:304] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.525650 28736 raft_consensus.cc:740] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.523331 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.526160 28736 consensus_queue.cc:260] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.526734 29842 raft_consensus.cc:2749] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:38.526795 28240 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: STOPPED
I20260320 15:27:38.526903 28240 leader_election.cc:304] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.527097 30010 raft_consensus.cc:2749] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.527465 28736 ts_tablet_manager.cc:1434] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:38.528208 28736 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.532251 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:38.532681 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:38.533038 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:38.533212 29640 ts_tablet_manager.cc:1929] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.533305 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:38.533412 29640 log.cc:1199] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7c4b53fb135741efad5d3e59a6d4eb3e
I20260320 15:27:38.533722 28890 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c] alter complete (version 2)
I20260320 15:27:38.533828 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 20eec704800e4d8b924d97556cc2700e
I20260320 15:27:38.533906 29640 ts_tablet_manager.cc:1950] T 7c4b53fb135741efad5d3e59a6d4eb3e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.534418 28901 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 71) took 4899 ms. Trace:
I20260320 15:27:38.534611 28901 rpcz_store.cc:276] 0320 15:27:33.635097 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.204762 (+3569665us) service_pool.cc:225] Handling call
0320 15:27:37.287237 (+ 82475us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:37.287417 (+ 180us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:37.287502 (+ 85us) catalog_manager.cc:2824] Locking tablets
0320 15:27:38.323263 (+1035761us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:38.331338 (+ 8075us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:38.331438 (+ 100us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:38.533750 (+202312us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:38.534374 (+ 624us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:38.324443 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.324589 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.324599 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:38.324603 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:38.324790 (+ 187us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.324812 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.324820 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:38.324989 (+ 169us) tablet.cc:700] Row locks acquired
0320 15:27:38.324994 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:38.325044 (+ 50us) write_op.cc:270] Start()
0320 15:27:38.325091 (+ 47us) write_op.cc:276] Timestamp: P: 1774020458325003 usec, L: 0
0320 15:27:38.325094 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.325399 (+ 305us) log.cc:844] Serialized 725 byte log entry
0320 15:27:38.330358 (+ 4959us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.330489 (+ 131us) write_op.cc:301] APPLY: starting
0320 15:27:38.330510 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.330566 (+ 56us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.330594 (+ 28us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.330674 (+ 80us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.330713 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:38.330720 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:38.330802 (+ 82us) log.cc:844] Serialized 29 byte log entry
0320 15:27:38.330869 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.330894 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:38.330918 (+ 24us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:38.330993 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.330997 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":499,"num_ops":2,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":1090,"prepare.run_wall_time_us":1090,"raft.queue_time_us":43,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":359,"replication_time_us":5223}]]}
I20260320 15:27:38.535887 28733 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.536938 29678 ts_tablet_manager.cc:1929] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.537065 29678 log.cc:1199] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:38.537446 29678 ts_tablet_manager.cc:1950] T 591ccc1d349e4fd599d5e343f91defe2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.538064 28733 tablet_bootstrap.cc:654] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.538311 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 107) took 4469 ms. Trace:
I20260320 15:27:38.538400 29678 rpcz_store.cc:276] 0320 15:27:34.068383 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.068841 (+ 458us) service_pool.cc:225] Handling call
0320 15:27:38.442176 (+4373335us) ts_tablet_manager.cc:1151] Deleting tablet 591ccc1d349e4fd599d5e343f91defe2
0320 15:27:38.442187 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.446849 (+ 4662us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.536807 (+ 89958us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.538287 (+ 1480us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4368368}
I20260320 15:27:38.538770 29678 tablet_replica.cc:333] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.538955 29678 raft_consensus.cc:2243] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.539094 29678 raft_consensus.cc:2272] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.540904 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 591ccc1d349e4fd599d5e343f91defe2 (table default.Table_ce377d5188904a50803bb93b251ac6aa [id=ab1248ac3aab491b8dd0fc75d85cef24]) successfully deleted
I20260320 15:27:38.541092 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:38.541802 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:38.541803 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:38.543715 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 99) took 4773 ms. Trace:
I20260320 15:27:38.543830 29640 rpcz_store.cc:276] 0320 15:27:33.769944 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.769998 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:38.271833 (+4501835us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:38.271847 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.472214 (+200367us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.533032 (+ 60818us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.543688 (+ 10656us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4491965}
I20260320 15:27:38.544241 29640 tablet_replica.cc:333] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.544481 29640 raft_consensus.cc:2243] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.544456 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7c4b53fb135741efad5d3e59a6d4eb3e (table default.Table_3796f2b714f64f18a0db9057941513ff [id=dede327e43e74b34b23d244805c6d1fe]) successfully deleted
I20260320 15:27:38.544672 29640 raft_consensus.cc:2272] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.545305 29640 ts_tablet_manager.cc:1916] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.549080 28733 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.549248 28733 ts_tablet_manager.cc:1403] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.002s
W20260320 15:27:38.549468 28239 consensus_peers.cc:597] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.550199 28733 raft_consensus.cc:359] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.550405 28733 raft_consensus.cc:385] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.550465 28733 raft_consensus.cc:740] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.550665 28733 consensus_queue.cc:260] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.551218 28733 ts_tablet_manager.cc:1434] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:38.551577 28733 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.552115 29676 consensus_queue.cc:1048] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.553486 28733 tablet_bootstrap.cc:654] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.557750 28736 tablet_bootstrap.cc:654] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.525157 29643 ts_tablet_manager.cc:1929] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.561751 29643 log.cc:1199] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ac4c7dc6859048ce956927201df44a86
I20260320 15:27:38.562454 29643 ts_tablet_manager.cc:1950] T ac4c7dc6859048ce956927201df44a86 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:38.563931 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 98) took 4486 ms. Trace:
I20260320 15:27:38.565338 29640 ts_tablet_manager.cc:1929] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.565476 29640 log.cc:1199] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/3e7b964395f24e45bc2715be3fac6dbe
I20260320 15:27:38.565822 29640 ts_tablet_manager.cc:1950] T 3e7b964395f24e45bc2715be3fac6dbe P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.566201 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 98) took 4796 ms. Trace:
I20260320 15:27:38.566301 29640 rpcz_store.cc:276] 0320 15:27:33.769562 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.769793 (+ 231us) service_pool.cc:225] Handling call
0320 15:27:38.544187 (+4774394us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:38.544198 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.561165 (+ 16967us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.565201 (+ 4036us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.566181 (+ 980us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4763885}
I20260320 15:27:38.566897 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 3e7b964395f24e45bc2715be3fac6dbe (table default.Table_c783052b0eb045ea95d77f3722e1ca8b [id=eba1ff6618e0486898840ab0fd2f0882]) successfully deleted
I20260320 15:27:38.568112 28446 raft_consensus.cc:1275] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Refusing update from remote peer 3ce5383d28b347ab979765b880e5773c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260320 15:27:38.569118 30010 consensus_queue.cc:1048] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260320 15:27:38.565110 28734 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.569590 28734 ts_tablet_manager.cc:1403] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.112s user 0.002s sys 0.002s
I20260320 15:27:38.572752 30010 raft_consensus.cc:493] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.572933 30010 raft_consensus.cc:515] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.573606 30010 leader_election.cc:290] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.574223 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.574275 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.574450 28441 raft_consensus.cc:2468] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.574461 28574 raft_consensus.cc:2468] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.574959 28239 leader_election.cc:304] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.575415 30010 raft_consensus.cc:2804] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.575603 30010 raft_consensus.cc:493] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.575682 30010 raft_consensus.cc:3060] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.578374 30010 raft_consensus.cc:515] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.565666 29643 rpcz_store.cc:276] 0320 15:27:34.077508 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.077563 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:38.325751 (+4248188us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:38.325766 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.444669 (+118903us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.525018 (+ 80349us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.563910 (+ 38892us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4246453}
I20260320 15:27:38.580219 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ac4c7dc6859048ce956927201df44a86 (table default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]) successfully deleted
I20260320 15:27:38.580740 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:38.580911 28574 raft_consensus.cc:3060] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.586508 28574 raft_consensus.cc:2468] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.588039 28239 leader_election.cc:304] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.588425 29618 raft_consensus.cc:2804] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.588596 29618 raft_consensus.cc:697] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.588841 29618 consensus_queue.cc:237] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.589872 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20a7164db1e0447fb424d8e46b747f52" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.590054 28441 raft_consensus.cc:3060] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.591245 30010 leader_election.cc:290] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.591764 29640 tablet_replica.cc:333] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.592020 29640 raft_consensus.cc:2243] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.592223 29640 raft_consensus.cc:2272] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.592809 28441 raft_consensus.cc:2468] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.592825 29640 ts_tablet_manager.cc:1916] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.592958 28733 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.593066 28733 ts_tablet_manager.cc:1403] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.000s
I20260320 15:27:38.593874 28733 raft_consensus.cc:359] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.594082 28733 raft_consensus.cc:385] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.594138 28733 raft_consensus.cc:740] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.594936 29678 ts_tablet_manager.cc:1916] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.595350 28733 consensus_queue.cc:260] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.600898 28733 ts_tablet_manager.cc:1434] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260320 15:27:38.605592 28733 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.606582 29643 tablet_replica.cc:333] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:38.606818 29643 raft_consensus.cc:2243] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.607004 29643 raft_consensus.cc:2272] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.607785 29643 ts_tablet_manager.cc:1916] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.611321 28734 raft_consensus.cc:359] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.611719 28734 raft_consensus.cc:385] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.611812 28734 raft_consensus.cc:740] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.612417 28734 consensus_queue.cc:260] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.613274 28734 ts_tablet_manager.cc:1434] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.042s user 0.002s sys 0.000s
I20260320 15:27:38.614205 29640 ts_tablet_manager.cc:1929] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.614351 29640 log.cc:1199] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:38.614835 29640 ts_tablet_manager.cc:1950] T b82c47ae3786401d931bb0b486ca3570 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.615871 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 100) took 4831 ms. Trace:
I20260320 15:27:38.615970 29640 rpcz_store.cc:276] 0320 15:27:33.785073 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.785136 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:38.591681 (+4806545us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:38.591700 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.596336 (+ 4636us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.614063 (+ 17727us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.615841 (+ 1778us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4805442}
I20260320 15:27:38.616537 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b82c47ae3786401d931bb0b486ca3570 (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
I20260320 15:27:38.617079 29640 tablet_replica.cc:333] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.617357 29640 raft_consensus.cc:2243] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.617609 29640 raft_consensus.cc:2272] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.618294 29640 ts_tablet_manager.cc:1916] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.624907 29678 ts_tablet_manager.cc:1929] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.625080 29678 log.cc:1199] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:38.625506 29678 ts_tablet_manager.cc:1950] T e2586d40343b4365ab8a67cce2e1c6ca P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.626343 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 108) took 4556 ms. Trace:
I20260320 15:27:38.626456 29678 rpcz_store.cc:276] 0320 15:27:34.070310 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.070360 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:38.538721 (+4468361us) ts_tablet_manager.cc:1151] Deleting tablet e2586d40343b4365ab8a67cce2e1c6ca
0320 15:27:38.538731 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.598930 (+ 60199us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.624750 (+ 25820us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.626317 (+ 1567us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4458942}
I20260320 15:27:38.627022 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet e2586d40343b4365ab8a67cce2e1c6ca (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
I20260320 15:27:38.634089 29640 ts_tablet_manager.cc:1929] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.636474 28734 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.647652 29640 log.cc:1199] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/66a9ddea49e641da881fba339ddef549
I20260320 15:27:38.648213 29640 ts_tablet_manager.cc:1950] T 66a9ddea49e641da881fba339ddef549 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.661080 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 101) took 4875 ms. Trace:
I20260320 15:27:38.657166 28734 tablet_bootstrap.cc:654] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.661342 29640 rpcz_store.cc:276] 0320 15:27:33.785728 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.785782 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:38.617010 (+4831228us) ts_tablet_manager.cc:1151] Deleting tablet 66a9ddea49e641da881fba339ddef549
0320 15:27:38.617021 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.629306 (+ 12285us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.633931 (+ 4625us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.661043 (+ 27112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4828983}
I20260320 15:27:38.662153 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 66a9ddea49e641da881fba339ddef549 (table default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]) successfully deleted
I20260320 15:27:38.662643 29640 tablet_replica.cc:333] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:38.664844 28383 consensus_peers.cc:597] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea95b7a1b30e42cebcbfcd551c58f679. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.666312 29678 tablet_replica.cc:333] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.666409 29618 raft_consensus.cc:493] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.666685 29618 raft_consensus.cc:515] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.668150 29618 leader_election.cc:290] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.668591 29678 raft_consensus.cc:2243] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.668963 28736 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.669327 29678 raft_consensus.cc:2272] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.671365 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.671725 29678 ts_tablet_manager.cc:1916] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.662910 29640 raft_consensus.cc:2243] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.672016 29640 raft_consensus.cc:2272] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.672633 29640 ts_tablet_manager.cc:1916] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.671648 28574 raft_consensus.cc:2468] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.675307 28733 tablet_bootstrap.cc:654] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.674777 28239 leader_election.cc:304] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.676414 29618 raft_consensus.cc:2804] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.676620 29618 raft_consensus.cc:493] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.676779 29618 raft_consensus.cc:3060] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.680629 29618 raft_consensus.cc:515] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.683562 29676 raft_consensus.cc:493] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.683753 29676 raft_consensus.cc:515] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.684569 29676 leader_election.cc:290] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.685539 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.685840 28446 raft_consensus.cc:2468] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.686417 28524 leader_election.cc:304] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.686851 29676 raft_consensus.cc:2804] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.686950 29676 raft_consensus.cc:493] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.687026 29676 raft_consensus.cc:3060] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.690516 29640 ts_tablet_manager.cc:1929] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.690443 29676 raft_consensus.cc:515] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.690646 29640 log.cc:1199] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/4becc525b59d421caf79d8cd42052118
I20260320 15:27:38.690984 29640 ts_tablet_manager.cc:1950] T 4becc525b59d421caf79d8cd42052118 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.691172 29676 leader_election.cc:290] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.692902 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.693240 28446 raft_consensus.cc:3060] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.693746 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 103) took 4905 ms. Trace:
I20260320 15:27:38.693861 29640 rpcz_store.cc:276] 0320 15:27:33.788673 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.788726 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:38.662576 (+4873850us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:38.662592 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.686166 (+ 23574us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.690369 (+ 4203us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.693719 (+ 3350us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4872370}
I20260320 15:27:38.694461 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 4becc525b59d421caf79d8cd42052118 (table default.Table_f6315615237b426ca7b2aea87662ecc9 [id=6097743ca3314c378c40adea38c460cc]) successfully deleted
I20260320 15:27:38.694871 29640 tablet_replica.cc:333] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.695130 29640 raft_consensus.cc:2243] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.695469 29640 raft_consensus.cc:2272] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.696162 29640 ts_tablet_manager.cc:1916] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.696808 28446 raft_consensus.cc:2468] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.697564 28524 leader_election.cc:304] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.697988 29676 raft_consensus.cc:2804] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.698287 29676 raft_consensus.cc:697] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.698635 29676 consensus_queue.cc:237] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.701260 29643 ts_tablet_manager.cc:1929] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.701514 29643 log.cc:1199] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:38.669530 28736 ts_tablet_manager.cc:1403] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.141s user 0.003s sys 0.010s
I20260320 15:27:38.685549 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.692662 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:38.709910 28524 leader_election.cc:343] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.709945 28736 raft_consensus.cc:359] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.710145 28736 raft_consensus.cc:385] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.710196 28736 raft_consensus.cc:740] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:38.710407 28524 leader_election.cc:343] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.710415 28736 consensus_queue.cc:260] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.710945 28736 ts_tablet_manager.cc:1434] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.711824 29618 leader_election.cc:290] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.713351 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:38.713555 28584 raft_consensus.cc:3060] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.717041 28584 raft_consensus.cc:2468] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
W20260320 15:27:38.700023 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 54265.5us
I20260320 15:27:38.717567 28239 leader_election.cc:304] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.718164 30018 raft_consensus.cc:2804] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.718302 28736 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:38.718437 30018 raft_consensus.cc:697] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.718748 30018 consensus_queue.cc:237] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:38.719244 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.719523 28444 raft_consensus.cc:3060] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.692024 30114 raft_consensus.cc:493] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.722821 28444 raft_consensus.cc:2468] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.723163 30114 raft_consensus.cc:515] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.723752 29640 ts_tablet_manager.cc:1929] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.723881 29640 log.cc:1199] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6231fe30c5b34d0bb7a233abbffeb1e8
I20260320 15:27:38.724152 30010 raft_consensus.cc:493] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.724292 29640 ts_tablet_manager.cc:1950] T 6231fe30c5b34d0bb7a233abbffeb1e8 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.724298 30010 raft_consensus.cc:515] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.724608 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f59a3c9fa17a4babaa9a18f18d51f193" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.724669 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 102) took 4937 ms. Trace:
I20260320 15:27:38.724862 28446 raft_consensus.cc:2376] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1. Re-sending same reply.
I20260320 15:27:38.724756 29640 rpcz_store.cc:276] 0320 15:27:33.786933 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.787016 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:38.694808 (+4907792us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:38.694823 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.708637 (+ 13814us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.723600 (+ 14963us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.724648 (+ 1048us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4904366}
I20260320 15:27:38.724944 30010 leader_election.cc:290] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.725211 30114 leader_election.cc:290] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.725378 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 6231fe30c5b34d0bb7a233abbffeb1e8 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]) successfully deleted
I20260320 15:27:38.726022 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.726114 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.726187 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a17aa9eba95d463b9c0fe5ae7e3cfe13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.726440 28441 raft_consensus.cc:2393] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:38.726584 28584 raft_consensus.cc:2468] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.726910 28239 leader_election.cc:304] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.727180 28240 leader_election.cc:304] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.728137 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.728399 28446 raft_consensus.cc:2468] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.728906 30010 raft_consensus.cc:2804] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.729002 30010 raft_consensus.cc:493] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.729089 30010 raft_consensus.cc:3060] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.729769 29618 raft_consensus.cc:3060] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.730491 28734 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.731781 30010 raft_consensus.cc:515] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.732420 30010 leader_election.cc:290] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.732715 29676 raft_consensus.cc:493] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.732918 29676 raft_consensus.cc:515] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.733057 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:38.733309 28584 raft_consensus.cc:3060] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.734577 29676 leader_election.cc:290] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.734853 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:38.735560 28524 leader_election.cc:343] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.736152 28734 ts_tablet_manager.cc:1403] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.000s
I20260320 15:27:38.736640 28584 raft_consensus.cc:2468] T 54a372bb37de43938f0420bdc9c7249c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.737114 28239 leader_election.cc:304] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:38.737401 30010 raft_consensus.cc:2804] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.737535 30010 raft_consensus.cc:697] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.737767 30010 consensus_queue.cc:237] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.734911 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.739048 28446 raft_consensus.cc:2468] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.739648 28524 leader_election.cc:304] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:38.739338 28736 tablet_bootstrap.cc:654] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.735884 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54a372bb37de43938f0420bdc9c7249c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.740288 28444 raft_consensus.cc:3060] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.742981 29640 tablet_replica.cc:333] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.743228 29640 raft_consensus.cc:2243] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.743418 29640 raft_consensus.cc:2272] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.744028 29640 ts_tablet_manager.cc:1916] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.744134 29617 raft_consensus.cc:2804] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.744284 29617 raft_consensus.cc:493] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.744369 29617 raft_consensus.cc:3060] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.745275 28733 tablet_bootstrap.cc:492] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.745663 28733 ts_tablet_manager.cc:1403] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.142s user 0.008s sys 0.000s
I20260320 15:27:38.748123 29617 raft_consensus.cc:515] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.747795 28733 raft_consensus.cc:359] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.748857 28733 raft_consensus.cc:385] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.750090 29617 leader_election.cc:290] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.750309 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:38.750244 28733 raft_consensus.cc:740] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:38.750895 28524 leader_election.cc:343] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.752703 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.753855 28446 raft_consensus.cc:3060] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.757308 28444 raft_consensus.cc:2468] T 54a372bb37de43938f0420bdc9c7249c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.732208 29643 ts_tablet_manager.cc:1950] T bb7a910bd2904a40ae63608377206c7e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:38.758702 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 99) took 4680 ms. Trace:
I20260320 15:27:38.758867 29643 rpcz_store.cc:276] 0320 15:27:34.078187 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.078241 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:38.606518 (+4528277us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:38.606530 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.654124 (+ 47594us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.701109 (+ 46985us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.758683 (+ 57574us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4528016}
I20260320 15:27:38.759505 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet bb7a910bd2904a40ae63608377206c7e (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
I20260320 15:27:38.756876 28733 consensus_queue.cc:260] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.760052 29643 tablet_replica.cc:333] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:38.760330 29643 raft_consensus.cc:2243] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:38.760466 28239 consensus_peers.cc:597] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e7b964395f24e45bc2715be3fac6dbe. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.769426 29640 ts_tablet_manager.cc:1929] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.769570 29640 log.cc:1199] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:38.766036 28733 ts_tablet_manager.cc:1434] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.020s user 0.004s sys 0.000s
I20260320 15:27:38.769922 29640 ts_tablet_manager.cc:1950] T 093ae84740d5468188b4a56fa16ded9c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.770845 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 105) took 4966 ms. Trace:
I20260320 15:27:38.770957 29640 rpcz_store.cc:276] 0320 15:27:33.804864 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.804922 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:38.735667 (+4930745us) ts_tablet_manager.cc:1151] Deleting tablet 093ae84740d5468188b4a56fa16ded9c
0320 15:27:38.735680 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.756032 (+ 20352us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.769298 (+ 13266us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.770823 (+ 1525us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4927499}
I20260320 15:27:38.771549 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 093ae84740d5468188b4a56fa16ded9c (table default.Table_20ceb59162bc456f86478ad67ada6956 [id=e79e8aae444f489eb30ffa5b6237e966]) successfully deleted
I20260320 15:27:38.772044 29640 tablet_replica.cc:333] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.772337 29640 raft_consensus.cc:2243] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.773134 28733 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.776296 29676 consensus_queue.cc:1048] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.780221 29618 raft_consensus.cc:2749] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.761005 29643 raft_consensus.cc:2272] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.781384 29643 ts_tablet_manager.cc:1916] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.787230 28736 tablet_bootstrap.cc:492] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:38.787710 28736 ts_tablet_manager.cc:1403] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.070s user 0.006s sys 0.001s
I20260320 15:27:38.789927 28736 raft_consensus.cc:359] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.790423 28736 raft_consensus.cc:385] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.790573 28736 raft_consensus.cc:740] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.791049 28736 consensus_queue.cc:260] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.792646 28733 tablet_bootstrap.cc:654] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.793816 28736 ts_tablet_manager.cc:1434] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260320 15:27:38.795660 28736 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:38.801362 28524 consensus_peers.cc:597] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcec6175c1ca49f5b682a29d5c1e235a. This is attempt 6: this message will repeat every 5th retry.
I20260320 15:27:38.808954 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:38.767916 28446 raft_consensus.cc:2468] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.811906 28524 leader_election.cc:304] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:38.812448 29617 raft_consensus.cc:2804] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.812628 29617 raft_consensus.cc:697] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.812940 29617 consensus_queue.cc:237] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.768576 28734 raft_consensus.cc:359] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.814502 28734 raft_consensus.cc:385] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.814707 28734 raft_consensus.cc:740] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.815302 28734 consensus_queue.cc:260] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.816632 29676 raft_consensus.cc:493] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.819581 28734 ts_tablet_manager.cc:1434] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.081s user 0.002s sys 0.000s
I20260320 15:27:38.820557 28734 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.822958 30018 consensus_queue.cc:1048] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.823709 29676 raft_consensus.cc:515] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.827888 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.828146 28443 raft_consensus.cc:2468] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.828687 28524 leader_election.cc:304] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.829071 29617 raft_consensus.cc:2804] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.829196 29617 raft_consensus.cc:493] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.829332 29617 raft_consensus.cc:3060] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.832155 29617 raft_consensus.cc:515] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.833050 29617 leader_election.cc:290] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.833606 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.833808 28443 raft_consensus.cc:3060] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.833874 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.834297 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5beaa1bc1e4465be34c8e86f131559" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:38.834520 28524 leader_election.cc:343] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:38.835484 28524 leader_election.cc:343] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.836560 28734 tablet_bootstrap.cc:654] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.836642 28443 raft_consensus.cc:2468] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.839701 28524 leader_election.cc:304] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:38.840515 29676 leader_election.cc:290] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:38.842109 28239 consensus_peers.cc:597] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:38.845628 28733 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.845783 28733 ts_tablet_manager.cc:1403] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.003s
I20260320 15:27:38.846572 28733 raft_consensus.cc:359] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.846745 28733 raft_consensus.cc:385] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.846802 28733 raft_consensus.cc:740] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.847007 28733 consensus_queue.cc:260] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.847703 28733 ts_tablet_manager.cc:1434] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.847993 28733 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.850172 28733 tablet_bootstrap.cc:654] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.851776 29678 ts_tablet_manager.cc:1929] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:38.844810 29676 raft_consensus.cc:2804] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.854239 29678 log.cc:1199] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:38.855266 28734 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.855736 29678 ts_tablet_manager.cc:1950] T 22b7f757ebdf456eb81085e5be54da4b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:38.855952 28734 ts_tablet_manager.cc:1403] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260320 15:27:38.857928 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:38.858392 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db81df6d3e9d49efb94c4f24de9db429 (table default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677]). Expected version 1 got 0
I20260320 15:27:38.858721 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:38.858978 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:38.859288 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:38.839798 29640 raft_consensus.cc:2272] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.859973 29599 consensus_queue.cc:1048] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:38.860131 29640 ts_tablet_manager.cc:1916] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.862174 28734 raft_consensus.cc:359] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.861680 29676 raft_consensus.cc:697] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.863267 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 110) took 4778 ms. Trace:
I20260320 15:27:38.864806 30018 raft_consensus.cc:493] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.865015 30018 raft_consensus.cc:515] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.865898 30018 leader_election.cc:290] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:38.867113 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:38.867746 28239 leader_election.cc:343] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.866446 29676 consensus_queue.cc:237] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.870352 28734 raft_consensus.cc:385] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.870527 28734 raft_consensus.cc:740] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.870168 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.870458 28736 tablet_bootstrap.cc:654] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.871280 28444 raft_consensus.cc:2468] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.871855 28240 leader_election.cc:304] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.872392 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:38.872607 29618 raft_consensus.cc:2804] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.872756 29618 raft_consensus.cc:493] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.872921 29618 raft_consensus.cc:3060] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.874013 29998 raft_consensus.cc:493] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.874158 29998 raft_consensus.cc:515] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.874794 29998 leader_election.cc:290] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.876932 29617 raft_consensus.cc:493] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.877058 29617 raft_consensus.cc:515] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.877342 28733 tablet_bootstrap.cc:492] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.877477 28733 ts_tablet_manager.cc:1403] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260320 15:27:38.878305 28733 raft_consensus.cc:359] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.878460 28733 raft_consensus.cc:385] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.877247 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.878515 28733 raft_consensus.cc:740] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.878687 28733 consensus_queue.cc:260] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:38.878963 28383 leader_election.cc:343] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.879115 28733 ts_tablet_manager.cc:1434] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:38.880259 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:38.881820 29618 raft_consensus.cc:515] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.882378 29998 raft_consensus.cc:493] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.882536 29998 raft_consensus.cc:515] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:38.882921 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.883074 28444 raft_consensus.cc:3060] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.883091 29998 leader_election.cc:290] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:38.884254 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:38.884872 28383 leader_election.cc:343] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.885906 28444 raft_consensus.cc:2468] T f05205970cd249b68d138b904ba61c9e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.870499 29678 rpcz_store.cc:276] 0320 15:27:34.085003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.085318 (+ 315us) service_pool.cc:225] Handling call
0320 15:27:38.626773 (+4541455us) ts_tablet_manager.cc:1151] Deleting tablet 22b7f757ebdf456eb81085e5be54da4b
0320 15:27:38.626786 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.738099 (+111313us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.851637 (+113538us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.863235 (+ 11598us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":280,"tablet-delete.queue_time_us":4530962}
I20260320 15:27:38.888358 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:38.888471 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 22b7f757ebdf456eb81085e5be54da4b (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
I20260320 15:27:38.888695 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:38.888914 29678 tablet_replica.cc:333] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:38.889094 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:38.889438 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:38.889773 29678 raft_consensus.cc:2243] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:38.889894 28524 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:38.890254 29678 raft_consensus.cc:2272] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.890415 29617 leader_election.cc:290] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.890486 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.890669 28304 raft_consensus.cc:2468] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:38.890985 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:38.891141 29678 ts_tablet_manager.cc:1916] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.891158 28524 leader_election.cc:304] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.891772 29599 raft_consensus.cc:2804] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.891875 29599 raft_consensus.cc:493] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.891953 29599 raft_consensus.cc:3060] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.892062 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:38.887861 28734 consensus_queue.cc:260] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.893591 28734 ts_tablet_manager.cc:1434] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.034s user 0.002s sys 0.000s
I20260320 15:27:38.893919 28734 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:38.894781 29599 raft_consensus.cc:515] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.891760 29643 ts_tablet_manager.cc:1929] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.896592 29643 log.cc:1199] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f70530cd63a0420fb51527759c951e31
I20260320 15:27:38.895470 29599 leader_election.cc:290] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:38.896705 28524 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:38.892798 28240 leader_election.cc:304] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:38.898367 30018 raft_consensus.cc:2804] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.898535 30018 raft_consensus.cc:697] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.898792 30018 consensus_queue.cc:237] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.896595 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac4c7dc6859048ce956927201df44a86" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:38.899730 28304 raft_consensus.cc:3060] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.902926 28304 raft_consensus.cc:2468] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:38.903376 28524 leader_election.cc:304] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:38.903908 29617 raft_consensus.cc:2804] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.904060 29617 raft_consensus.cc:697] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:38.904376 30010 raft_consensus.cc:493] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.904340 29617 consensus_queue.cc:237] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.904621 30010 raft_consensus.cc:515] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.905313 30010 leader_election.cc:290] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:38.906810 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:38.906941 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.907022 28441 raft_consensus.cc:2468] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:38.907228 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.907496 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.907887 29640 ts_tablet_manager.cc:1929] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.908000 29640 log.cc:1199] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2165371f4b5448c7a2625926082ee4c6
W20260320 15:27:38.907955 28239 leader_election.cc:343] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:38.908054 28382 leader_election.cc:343] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.908167 28382 leader_election.cc:304] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.908386 29640 ts_tablet_manager.cc:1950] T 2165371f4b5448c7a2625926082ee4c6 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.909020 29842 raft_consensus.cc:2749] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.909053 28733 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:38.909574 28382 leader_election.cc:343] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.909677 28382 leader_election.cc:304] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.909994 29840 raft_consensus.cc:2749] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:38.910576 28733 tablet_bootstrap.cc:654] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.910581 28240 leader_election.cc:304] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.911023 30114 raft_consensus.cc:2804] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:38.911109 30114 raft_consensus.cc:493] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:38.911183 30114 raft_consensus.cc:3060] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.913904 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:38.914036 30114 raft_consensus.cc:515] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.914131 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:38.914348 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:38.914743 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:38.915088 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:38.915354 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:38.915628 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:38.914688 30114 leader_election.cc:290] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:38.916040 28239 leader_election.cc:343] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.916625 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 104) took 5114 ms. Trace:
I20260320 15:27:38.916728 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:38.916910 28441 raft_consensus.cc:3060] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:38.920060 28441 raft_consensus.cc:2468] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:38.916723 29640 rpcz_store.cc:276] 0320 15:27:33.802496 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.802983 (+ 487us) service_pool.cc:225] Handling call
0320 15:27:38.771957 (+4968974us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:38.771971 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.863388 (+ 91417us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.907764 (+ 44376us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.916605 (+ 8841us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4956555}
I20260320 15:27:38.920895 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 2165371f4b5448c7a2625926082ee4c6 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
I20260320 15:27:38.920627 28240 leader_election.cc:304] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.921254 29640 tablet_replica.cc:333] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.921520 29640 raft_consensus.cc:2243] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.921659 30114 raft_consensus.cc:2804] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:38.921916 29640 raft_consensus.cc:2272] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.921950 30114 raft_consensus.cc:697] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:38.922267 30114 consensus_queue.cc:237] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.922602 29640 ts_tablet_manager.cc:1916] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.925199 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:38.928386 28733 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:38.928552 28733 ts_tablet_manager.cc:1403] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.005s
I20260320 15:27:38.929383 28733 raft_consensus.cc:359] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.929575 28733 raft_consensus.cc:385] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:38.929633 28733 raft_consensus.cc:740] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:38.929865 28733 consensus_queue.cc:260] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.930506 28733 ts_tablet_manager.cc:1434] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:38.935904 28733 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:38.936641 29643 ts_tablet_manager.cc:1950] T f70530cd63a0420fb51527759c951e31 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:38.937675 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f05205970cd249b68d138b904ba61c9e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:38.938198 28239 leader_election.cc:343] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.938511 29618 leader_election.cc:290] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:38.939291 28734 tablet_bootstrap.cc:654] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.939225 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:38.939541 28901 catalog_manager.cc:5671] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:38.939997 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:38.940290 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:38.941282 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:38.941716 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:38.942045 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:38.942493 28901 catalog_manager.cc:5671] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:38.942999 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:38.943346 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:38.941161 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 100) took 4862 ms. Trace:
W20260320 15:27:38.945472 28873 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 80810790e42947a69ce6140feb9f7cbc consensus state has no leader
I20260320 15:27:38.945173 29643 rpcz_store.cc:276] 0320 15:27:34.078891 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.079027 (+ 136us) service_pool.cc:225] Handling call
0320 15:27:38.759981 (+4680954us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:38.759995 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.784692 (+ 24697us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.891611 (+106919us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.941092 (+ 49481us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4680669}
I20260320 15:27:38.951988 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet f70530cd63a0420fb51527759c951e31 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:38.952430 28733 tablet_bootstrap.cc:654] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:38.955778 29643 tablet_replica.cc:333] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:38.956836 29643 raft_consensus.cc:2243] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:38.956833 29640 ts_tablet_manager.cc:1929] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:38.956969 29640 log.cc:1199] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:38.957414 29640 ts_tablet_manager.cc:1950] T 993e2fb1561f4317be85e9b5e5816edc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:38.957794 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 107) took 5134 ms. Trace:
I20260320 15:27:38.957895 29640 rpcz_store.cc:276] 0320 15:27:33.823385 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.823436 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:38.921187 (+5097751us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:38.921202 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.946553 (+ 25351us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.956691 (+ 10138us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:38.957775 (+ 1084us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5079952}
I20260320 15:27:38.958390 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 993e2fb1561f4317be85e9b5e5816edc (table default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]) successfully deleted
I20260320 15:27:38.957141 29842 raft_consensus.cc:493] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:38.960296 29842 raft_consensus.cc:515] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.960470 29643 raft_consensus.cc:2272] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:38.960988 29842 leader_election.cc:290] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:38.961064 29643 ts_tablet_manager.cc:1916] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:38.962117 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:38.962703 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:38.962695 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:38.963218 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
W20260320 15:27:38.963337 28383 leader_election.cc:343] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.963884 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:38.960065 28734 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:38.965471 28734 ts_tablet_manager.cc:1403] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.072s user 0.004s sys 0.000s
I20260320 15:27:38.965626 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:38.968185 28734 raft_consensus.cc:359] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:38.969659 28906 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_f355c2a83bf94395b0e1234d65483b00 [id=18c937cd7a8344e5a9c283fb8d303f76] alter complete (version 2)
W20260320 15:27:38.970036 28524 consensus_peers.cc:597] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f70530cd63a0420fb51527759c951e31. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:38.970270 28889 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 31379c9e2adc4fd48e0d0b6e9443615a consensus state has no leader
I20260320 15:27:38.970530 28889 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 73) took 5070 ms. Trace:
I20260320 15:27:38.970649 28892 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072] alter complete (version 1)
I20260320 15:27:38.971854 28902 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_2f931a99db714d77af814885516e04f7 [id=99887ef4c8074e5daff46d3f5614cbfe] alter complete (version 4)
W20260320 15:27:38.972152 28903 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bbc5d5f49a04216a352ddb1b571dce0 consensus state has no leader
I20260320 15:27:38.972427 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 72) took 5085 ms. Trace:
I20260320 15:27:38.972527 28903 rpcz_store.cc:276] 0320 15:27:33.886940 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.219249 (+3332309us) service_pool.cc:225] Handling call
0320 15:27:37.219871 (+ 622us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:37.220165 (+ 294us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:37.272191 (+ 52026us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:37.272361 (+ 170us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:37.281225 (+ 8864us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:38.972347 (+1691122us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:37.273018 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:37.273211 (+ 193us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:37.273222 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:37.273226 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:37.273389 (+ 163us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:37.273412 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:37.273419 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:37.273516 (+ 97us) tablet.cc:700] Row locks acquired
0320 15:27:37.273518 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:37.273570 (+ 52us) write_op.cc:270] Start()
0320 15:27:37.273620 (+ 50us) write_op.cc:276] Timestamp: P: 1774020457273514 usec, L: 0
0320 15:27:37.273626 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:37.273897 (+ 271us) log.cc:844] Serialized 405 byte log entry
0320 15:27:37.279478 (+ 5581us) op_driver.cc:464] REPLICATION: finished
0320 15:27:37.279641 (+ 163us) write_op.cc:301] APPLY: starting
0320 15:27:37.279663 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:37.279709 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:37.279737 (+ 28us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:37.279801 (+ 64us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:37.279839 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:37.279846 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:37.279943 (+ 97us) log.cc:844] Serialized 23 byte log entry
0320 15:27:37.280058 (+ 115us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.280084 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:37.280091 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:37.280172 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.280175 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":104,"apply.run_cpu_time_us":522,"apply.run_wall_time_us":522,"num_ops":1,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":1020,"prepare.run_wall_time_us":1019,"raft.queue_time_us":337,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":255,"replication_time_us":5815}]]}
I20260320 15:27:38.973649 28905 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_e26aaeaeae1047e8b3ea16c949628001 [id=b7fdfea8538d4e758051c53a61442e69] alter complete (version 2)
I20260320 15:27:38.975978 29640 tablet_replica.cc:333] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:38.976588 29640 raft_consensus.cc:2243] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:38.977152 29640 raft_consensus.cc:2272] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:38.977895 28382 leader_election.cc:343] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:38.978070 28382 leader_election.cc:304] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:38.970659 28889 rpcz_store.cc:276] 0320 15:27:33.900025 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.255329 (+3355304us) service_pool.cc:225] Handling call
0320 15:27:37.287612 (+ 32283us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:37.287817 (+ 205us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:37.288047 (+ 230us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:37.288130 (+ 83us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:37.298118 (+ 9988us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:38.970452 (+1672334us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:37.289993 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:37.290163 (+ 170us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:37.290174 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:37.290178 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:37.290354 (+ 176us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:37.290375 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:37.290383 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:37.290465 (+ 82us) tablet.cc:700] Row locks acquired
0320 15:27:37.290469 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:37.290512 (+ 43us) write_op.cc:270] Start()
0320 15:27:37.290550 (+ 38us) write_op.cc:276] Timestamp: P: 1774020457290459 usec, L: 0
0320 15:27:37.290554 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:37.290801 (+ 247us) log.cc:844] Serialized 405 byte log entry
0320 15:27:37.294471 (+ 3670us) op_driver.cc:464] REPLICATION: finished
0320 15:27:37.294589 (+ 118us) write_op.cc:301] APPLY: starting
0320 15:27:37.294608 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:37.294654 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:37.294693 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:37.294750 (+ 57us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:37.294785 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:37.294792 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:37.294879 (+ 87us) log.cc:844] Serialized 23 byte log entry
0320 15:27:37.294959 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.294983 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:37.294991 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:37.295101 (+ 110us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.295105 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":499,"num_ops":1,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":958,"prepare.run_wall_time_us":958,"raft.queue_time_us":89,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":277,"replication_time_us":3883}]]}
I20260320 15:27:38.979761 29840 raft_consensus.cc:2749] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:38.996516 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 118) took 1120 ms (client timeout 1000 ms). Trace:
W20260320 15:27:38.996709 28889 rpcz_store.cc:269] 0320 15:27:37.875696 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.979175 (+1103479us) service_pool.cc:212] Skipping call since client already timed out
0320 15:27:38.996400 (+ 17225us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260320 15:27:39.005359 29840 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.005560 29840 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.006294 29840 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.016899 29643 ts_tablet_manager.cc:1929] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.017463 29643 log.cc:1199] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:39.017802 29640 ts_tablet_manager.cc:1916] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.018968 29643 ts_tablet_manager.cc:1950] T ea95b7a1b30e42cebcbfcd551c58f679 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.019874 29617 raft_consensus.cc:493] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.020045 29617 raft_consensus.cc:515] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.032493 28905 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_42f7889a62f945acac082806f1cec3f4 [id=3120af75e43943d68a8cd8ef413acb4e] alter complete (version 1)
I20260320 15:27:39.043234 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 101) took 4963 ms. Trace:
I20260320 15:27:39.044833 29643 rpcz_store.cc:276] 0320 15:27:34.079808 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.079874 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:38.951802 (+4871928us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:38.951814 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.967604 (+ 15790us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.014877 (+ 47273us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.043165 (+ 28288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4871657}
I20260320 15:27:39.047626 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ea95b7a1b30e42cebcbfcd551c58f679 (table default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]) successfully deleted
I20260320 15:27:39.047854 29643 tablet_replica.cc:333] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.048120 29643 raft_consensus.cc:2243] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:39.047379 30128 log.cc:927] Time spent T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260320 15:27:39.048559 29643 raft_consensus.cc:2272] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.048658 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:39.051869 29643 ts_tablet_manager.cc:1916] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.052515 29617 leader_election.cc:290] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.051731 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.053901 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.054155 28444 raft_consensus.cc:2468] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:39.054773 28524 leader_election.cc:304] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.056021 29617 raft_consensus.cc:2804] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:39.056134 29617 raft_consensus.cc:493] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:39.056214 29617 raft_consensus.cc:3060] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.079167 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.080343 29678 ts_tablet_manager.cc:1929] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:39.080467 29678 log.cc:1199] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/fa22d207ad8a4555aa9573b15fa1fe1a
W20260320 15:27:39.081679 28524 leader_election.cc:343] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.084233 29678 ts_tablet_manager.cc:1950] T fa22d207ad8a4555aa9573b15fa1fe1a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:39.084777 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 111) took 4957 ms. Trace:
I20260320 15:27:39.084945 29678 rpcz_store.cc:276] 0320 15:27:34.127591 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.127675 (+ 84us) service_pool.cc:225] Handling call
0320 15:27:38.888250 (+4760575us) ts_tablet_manager.cc:1151] Deleting tablet fa22d207ad8a4555aa9573b15fa1fe1a
0320 15:27:38.888264 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:38.953987 (+ 65723us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.080223 (+126236us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.084759 (+ 4536us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4760272}
I20260320 15:27:39.085475 29678 tablet_replica.cc:333] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:39.085613 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet fa22d207ad8a4555aa9573b15fa1fe1a (table default.Table_2fdea31f3e72453d949a79a1daf3d9b6 [id=02a4dd16dc544e02b5f579361e462b84]) successfully deleted
I20260320 15:27:39.085774 29678 raft_consensus.cc:2243] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.085968 29678 raft_consensus.cc:2272] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.089946 29643 ts_tablet_manager.cc:1929] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.090121 29643 log.cc:1199] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:39.090674 29643 ts_tablet_manager.cc:1950] T 9a072859f39c4c28ae65b0e9afa79d66 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.090657 28736 tablet_bootstrap.cc:492] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:39.090876 30114 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.091043 29678 ts_tablet_manager.cc:1916] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.091028 30114 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.091094 28736 ts_tablet_manager.cc:1403] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.296s user 0.007s sys 0.003s
I20260320 15:27:39.092617 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.093783 30114 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:39.094976 28239 leader_election.cc:343] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 757b5afdea674217909610e3719a8804
I20260320 15:27:39.095144 28239 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:39.096038 30114 raft_consensus.cc:2749] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:39.100248 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 102) took 5019 ms. Trace:
I20260320 15:27:39.102195 29643 rpcz_store.cc:276] 0320 15:27:34.080465 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.080533 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:39.047761 (+4967228us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:39.047775 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.072976 (+ 25201us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.089783 (+ 16807us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.100193 (+ 10410us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4967006}
W20260320 15:27:39.101384 28382 consensus_peers.cc:597] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c4b53fb135741efad5d3e59a6d4eb3e. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.102937 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9a072859f39c4c28ae65b0e9afa79d66 (table default.Table_874d6a667bff4c3d815fcff90cae13f5 [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
I20260320 15:27:39.103307 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 103) took 5022 ms. Trace:
I20260320 15:27:39.103446 29643 rpcz_store.cc:276] 0320 15:27:34.081049 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.081103 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:39.102992 (+5021889us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:39.103007 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.103246 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5021651}
I20260320 15:27:39.104103 29643 tablet_replica.cc:333] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:39.104082 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
W20260320 15:27:39.104717 28382 leader_election.cc:343] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:39.104858 28382 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.106057 29840 raft_consensus.cc:2749] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:39.106370 30133 consensus_queue.cc:1048] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.106710 29643 raft_consensus.cc:2243] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.107215 29643 raft_consensus.cc:2272] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.108372 29643 ts_tablet_manager.cc:1916] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.115703 29617 raft_consensus.cc:515] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.119855 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:39.126659 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.127620 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.127890 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.128050 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.128196 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.128367 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.128554 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.128736 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:39.128808 28892 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet fca60a0f372f4d1194d754d124b78ff8 consensus state has no leader
I20260320 15:27:39.132750 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:39.133013 28441 raft_consensus.cc:3060] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.133183 29617 leader_election.cc:290] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:39.134789 28892 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e779a274913f4caeaf82e3f180457830 consensus state has no leader
W20260320 15:27:39.135175 28892 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet ef0d80754dfd4bbd9817f568063511ef consensus state has no leader
I20260320 15:27:39.143102 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.143433 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.148283 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.150226 28441 raft_consensus.cc:2468] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:39.152101 28524 leader_election.cc:304] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.152740 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:39.153842 28524 leader_election.cc:343] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.154314 29617 raft_consensus.cc:2804] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:39.154438 29617 raft_consensus.cc:697] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:39.154685 29617 consensus_queue.cc:237] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.155615 28904 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_22b004e387544471bf99f3f2f1267d63 [id=f9156a7fb60142b886e96708d989e7e2] alter complete (version 3)
I20260320 15:27:39.159667 29939 raft_consensus.cc:493] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.159849 29939 raft_consensus.cc:515] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:39.161681 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:39.163264 29939 leader_election.cc:290] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.164048 29678 ts_tablet_manager.cc:1929] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.164237 29678 log.cc:1199] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7394e941c99e4358ad7a33db84b15348
I20260320 15:27:39.164728 29676 consensus_queue.cc:1048] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.172008 29678 ts_tablet_manager.cc:1950] T 7394e941c99e4358ad7a33db84b15348 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:39.173349 28524 leader_election.cc:343] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: d547c032332d4d109222b4743fa28561
I20260320 15:27:39.173345 29643 ts_tablet_manager.cc:1929] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.173538 29643 log.cc:1199] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:39.173663 28524 leader_election.cc:304] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:39.173995 29617 raft_consensus.cc:2749] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.174561 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 109) took 5098 ms. Trace:
I20260320 15:27:39.174875 29678 rpcz_store.cc:276] 0320 15:27:34.076514 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.076894 (+ 380us) service_pool.cc:225] Handling call
0320 15:27:39.085417 (+5008523us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:39.085430 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.151417 (+ 65987us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.163920 (+ 12503us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.174542 (+ 10622us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4954532}
I20260320 15:27:39.175850 29678 tablet_replica.cc:333] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:39.176380 29678 raft_consensus.cc:2243] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.176965 29643 ts_tablet_manager.cc:1950] T 66a181681cd84b7fb09c3742f4f2222f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.177423 29678 raft_consensus.cc:2272] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.177403 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 104) took 5095 ms. Trace:
I20260320 15:27:39.178398 29643 rpcz_store.cc:276] 0320 15:27:34.081650 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.081708 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:39.104029 (+5022321us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:39.104039 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.116979 (+ 12940us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.173097 (+ 56118us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.177355 (+ 4258us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5022095}
I20260320 15:27:39.178534 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7394e941c99e4358ad7a33db84b15348 (table default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]) successfully deleted
I20260320 15:27:39.178786 29643 tablet_replica.cc:333] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.179013 29643 raft_consensus.cc:2243] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.179028 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 66a181681cd84b7fb09c3742f4f2222f (table default.Table_cfc52407bf394111b223a209b9adbdec [id=c827d7a27c8d413eb02be8f6c3ffb64b]) successfully deleted
I20260320 15:27:39.179231 29643 raft_consensus.cc:2272] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.179903 29643 ts_tablet_manager.cc:1916] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:39.179900 28892 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 101) took 4072 ms (client timeout 1000 ms). Trace:
W20260320 15:27:39.180038 28892 rpcz_store.cc:269] 0320 15:27:35.107908 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.590586 (+482678us) service_pool.cc:225] Handling call
0320 15:27:39.179842 (+3589256us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.887696 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.887893 (+ 197us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.887905 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:35.887909 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:35.888081 (+ 172us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.888097 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.888102 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:35.888292 (+ 190us) tablet.cc:700] Row locks acquired
0320 15:27:35.888297 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:35.888354 (+ 57us) write_op.cc:270] Start()
0320 15:27:35.888419 (+ 65us) write_op.cc:276] Timestamp: P: 1774020455888307 usec, L: 0
0320 15:27:35.888427 (+ 8us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.888749 (+ 322us) log.cc:844] Serialized 901 byte log entry
0320 15:27:35.899039 (+ 10290us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.899156 (+ 117us) write_op.cc:301] APPLY: starting
0320 15:27:35.899175 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.899227 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.899265 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.899337 (+ 72us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.899374 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:35.899382 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:35.899467 (+ 85us) log.cc:844] Serialized 29 byte log entry
0320 15:27:35.899564 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.899589 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:35.899596 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.899683 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.899686 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:35.973767 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.973967 (+ 200us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.973977 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:35.973981 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:35.974139 (+ 158us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.974163 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.974171 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:35.974355 (+ 184us) tablet.cc:700] Row locks acquired
0320 15:27:35.974360 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:35.974415 (+ 55us) write_op.cc:270] Start()
0320 15:27:35.974457 (+ 42us) write_op.cc:276] Timestamp: P: 1774020455974362 usec, L: 0
0320 15:27:35.974461 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.974757 (+ 296us) log.cc:844] Serialized 356 byte log entry
0320 15:27:35.979909 (+ 5152us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.980046 (+ 137us) write_op.cc:301] APPLY: starting
0320 15:27:35.980069 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.980127 (+ 58us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.980182 (+ 55us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.980287 (+ 105us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.980340 (+ 53us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:35.980351 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:35.980468 (+ 117us) log.cc:844] Serialized 23 byte log entry
0320 15:27:35.980554 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.980587 (+ 33us) write_op.cc:454] Released schema lock
0320 15:27:35.980598 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.980696 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.980703 (+ 7us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:37.148947 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:37.149068 (+ 121us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:37.149075 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:37.149078 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:37.149201 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:37.149218 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:37.149222 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:37.149296 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:37.149299 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:37.149337 (+ 38us) write_op.cc:270] Start()
0320 15:27:37.149380 (+ 43us) write_op.cc:276] Timestamp: P: 1774020457149297 usec, L: 0
0320 15:27:37.149384 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:37.149581 (+ 197us) log.cc:844] Serialized 356 byte log entry
0320 15:27:37.152602 (+ 3021us) op_driver.cc:464] REPLICATION: finished
0320 15:27:37.152734 (+ 132us) write_op.cc:301] APPLY: starting
0320 15:27:37.152755 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:37.152801 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:37.152856 (+ 55us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:37.152921 (+ 65us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:37.152959 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:37.152966 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:37.153045 (+ 79us) log.cc:844] Serialized 23 byte log entry
0320 15:27:37.153110 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.153133 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:37.153140 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:37.153216 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.153220 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":142049,"reported_tablets":207,"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":509,"apply.run_wall_time_us":509,"num_ops":2,"prepare.queue_time_us":264,"prepare.run_cpu_time_us":1217,"prepare.run_wall_time_us":1218,"raft.queue_time_us":605,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":375,"replication_time_us":10571}],["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":635,"apply.run_wall_time_us":634,"num_ops":1,"prepare.queue_time_us":310,"prepare.run_cpu_time_us":1159,"prepare.run_wall_time_us":1160,"raft.queue_time_us":787,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":375,"replication_time_us":5407}],["op",{"apply.queue_time_us":42,"apply.run_cpu_time_us":484,"apply.run_wall_time_us":485,"num_ops":1,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":755,"prepare.run_wall_time_us":758,"raft.queue_time_us":41,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":267,"replication_time_us":3181}]]}
I20260320 15:27:39.181665 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c883a162464b4030a2c38e1eb9b7efe4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.182318 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c883a162464b4030a2c38e1eb9b7efe4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.187664 29678 ts_tablet_manager.cc:1916] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.190725 30114 raft_consensus.cc:493] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.190899 30114 raft_consensus.cc:515] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.191524 30114 leader_election.cc:290] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:39.199389 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.199662 28441 raft_consensus.cc:2468] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:39.203490 28240 leader_election.cc:304] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.203783 29998 raft_consensus.cc:493] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.203994 29998 raft_consensus.cc:515] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.204685 30010 raft_consensus.cc:2804] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:39.204742 29998 leader_election.cc:290] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.204807 30010 raft_consensus.cc:493] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:39.205960 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:39.206604 28382 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:39.206492 28240 consensus_peers.cc:597] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.214921 29640 ts_tablet_manager.cc:1929] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:39.215102 29640 log.cc:1199] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:39.215557 29640 ts_tablet_manager.cc:1950] T 16e31e11048b494ebe1fd4d89488b0cf P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:39.215996 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 106) took 5395 ms. Trace:
I20260320 15:27:39.216112 29640 rpcz_store.cc:276] 0320 15:27:33.821837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.821903 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:38.975903 (+5154000us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:38.975920 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.198392 (+222472us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.214760 (+ 16368us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.215968 (+ 1208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5116220}
I20260320 15:27:39.216620 29640 tablet_replica.cc:333] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:39.216866 29640 raft_consensus.cc:2243] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.217334 29640 raft_consensus.cc:2272] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.218082 29640 ts_tablet_manager.cc:1916] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.222143 28733 tablet_bootstrap.cc:492] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.222301 28733 ts_tablet_manager.cc:1403] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.286s user 0.014s sys 0.016s
I20260320 15:27:39.223059 28733 raft_consensus.cc:359] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.223249 28733 raft_consensus.cc:385] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.223313 28733 raft_consensus.cc:740] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.223513 28733 consensus_queue.cc:260] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.224197 28733 ts_tablet_manager.cc:1434] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:38.969689 28734 raft_consensus.cc:385] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.225562 29678 ts_tablet_manager.cc:1929] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.226128 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 16e31e11048b494ebe1fd4d89488b0cf (table default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]) successfully deleted
I20260320 15:27:39.226100 29678 log.cc:1199] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:39.227833 29678 ts_tablet_manager.cc:1950] T b03e7555b9a646b599acb0c7dff90c9a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:39.229671 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 112) took 5101 ms. Trace:
I20260320 15:27:39.230477 29678 rpcz_store.cc:276] 0320 15:27:34.128083 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.128123 (+ 40us) service_pool.cc:225] Handling call
0320 15:27:39.175797 (+5047674us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:39.175809 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.204544 (+ 28735us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.224955 (+ 20411us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.229540 (+ 4585us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5043744}
I20260320 15:27:39.232748 28733 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:39.234584 28733 tablet_bootstrap.cc:654] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.234510 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 113) took 5106 ms. Trace:
I20260320 15:27:39.236719 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 093ae84740d5468188b4a56fa16ded9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.237498 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 9aa2d48a54dd403c82d7f3a64db5aeab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.235594 29678 rpcz_store.cc:276] 0320 15:27:34.128227 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.128278 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:39.233171 (+5104893us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:39.233186 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.234259 (+ 1073us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5099790}
I20260320 15:27:39.238070 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet b82c47ae3786401d931bb0b486ca3570 with delete_type TABLET_DATA_DELETED (replaced by tablet cc97a5f2bd964a05bb7b2427226debdd) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.239102 29678 tablet_replica.cc:333] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:39.240931 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b03e7555b9a646b599acb0c7dff90c9a (table default.Table_a894581517cb4471a20cffa8c3ec26f5 [id=e539cb674eb14059bdb2332a1d1f5bda]) successfully deleted
I20260320 15:27:39.241158 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:39.241389 28584 raft_consensus.cc:2468] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:39.240949 29678 raft_consensus.cc:2243] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.243698 29678 raft_consensus.cc:2272] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:39.246767 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 16e31e11048b494ebe1fd4d89488b0cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 16e31e11048b494ebe1fd4d89488b0cf already in progress: creating tablet
I20260320 15:27:39.240499 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.252278 28734 raft_consensus.cc:740] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.253273 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.253149 28734 consensus_queue.cc:260] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.256071 28734 ts_tablet_manager.cc:1434] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.290s user 0.004s sys 0.000s
W20260320 15:27:39.257301 28383 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.257373 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 2165371f4b5448c7a2625926082ee4c6 with delete_type TABLET_DATA_DELETED (replaced by tablet fa5618796f1a4ff3ab4c1209dad0aede) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.257501 28383 leader_election.cc:304] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.257802 29842 raft_consensus.cc:2749] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.258124 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.258565 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.258970 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.259389 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.259887 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.260289 28736 raft_consensus.cc:359] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.260582 28736 raft_consensus.cc:385] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.260690 28736 raft_consensus.cc:740] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:39.261613 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 102) took 2613 ms (client timeout 1000 ms). Trace:
I20260320 15:27:39.260970 28736 consensus_queue.cc:260] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.261430 29678 ts_tablet_manager.cc:1916] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.262233 28736 ts_tablet_manager.cc:1434] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.171s user 0.002s sys 0.000s
W20260320 15:27:39.268985 28890 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 7f94140a5e8b4a06890bca86456d64a6 (table default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7f94140a5e8b4a06890bca86456d64a6 consensus state has no leader
W20260320 15:27:39.265666 28904 rpcz_store.cc:269] 0320 15:27:36.632551 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.066893 (+434342us) service_pool.cc:225] Handling call
0320 15:27:39.245880 (+2178987us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.123791 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.123968 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.123976 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:39.123980 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:39.124119 (+ 139us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.124139 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.124147 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.124233 (+ 86us) tablet.cc:700] Row locks acquired
0320 15:27:39.124237 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:39.124287 (+ 50us) write_op.cc:270] Start()
0320 15:27:39.124336 (+ 49us) write_op.cc:276] Timestamp: P: 1774020459124262 usec, L: 0
0320 15:27:39.124341 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.124642 (+ 301us) log.cc:844] Serialized 356 byte log entry
0320 15:27:39.137326 (+ 12684us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.137484 (+ 158us) write_op.cc:301] APPLY: starting
0320 15:27:39.137510 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.137565 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.137612 (+ 47us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.137688 (+ 76us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.137735 (+ 47us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.137747 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:39.137895 (+ 148us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.137975 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.138004 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:39.138013 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.138109 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.138115 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103386,"reported_tablets":210,"child_traces":[["op",{"apply.queue_time_us":93,"apply.run_cpu_time_us":612,"apply.run_wall_time_us":611,"num_ops":1,"prepare.queue_time_us":6193,"prepare.run_cpu_time_us":1031,"prepare.run_wall_time_us":1032,"raft.queue_time_us":493,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":359,"replication_time_us":12931}]]}
I20260320 15:27:39.275095 29643 ts_tablet_manager.cc:1929] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.288326 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.288640 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.288772 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.288949 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.288640 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet fa22d207ad8a4555aa9573b15fa1fe1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:18 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.289287 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.289489 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet e2586d40343b4365ab8a67cce2e1c6ca with delete_type TABLET_DATA_DELETED (replaced by tablet 9dd293a68f894cbfa122d53918816c28) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.291976 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.292455 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.293318 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet b34048a722c3485490637965c60cc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.294658 28736 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:39.298979 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.299873 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.300498 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.301048 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.283070 29998 raft_consensus.cc:493] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.302074 29643 log.cc:1199] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8a18f32c29224bd883ceb20658dff228
I20260320 15:27:39.302083 29998 raft_consensus.cc:515] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.302573 29643 ts_tablet_manager.cc:1950] T 8a18f32c29224bd883ceb20658dff228 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.302886 29998 leader_election.cc:290] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.304037 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:39.304788 28382 leader_election.cc:343] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.299439 29842 raft_consensus.cc:493] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.305294 29842 raft_consensus.cc:515] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.306008 29842 leader_election.cc:290] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.306861 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:39.307557 28382 leader_election.cc:343] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.319401 29678 ts_tablet_manager.cc:1929] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:39.320261 29678 log.cc:1199] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:39.322048 29678 ts_tablet_manager.cc:1950] T adb65832eefe4bd4b08d95aa83c7bd7b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:39.322228 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.323246 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 931f6f88a1d94a40b69f07216b8313c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.325944 28734 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:39.326351 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 105) took 5244 ms. Trace:
I20260320 15:27:39.326489 29643 rpcz_store.cc:276] 0320 15:27:34.082214 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.082263 (+ 49us) service_pool.cc:225] Handling call
0320 15:27:39.178707 (+5096444us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:39.178718 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.190208 (+ 11490us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.274904 (+ 84696us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.326297 (+ 51393us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5096222}
I20260320 15:27:39.327314 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.327437 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:39.328459 28383 leader_election.cc:343] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.328617 28383 leader_election.cc:304] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.329128 29998 raft_consensus.cc:2749] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:39.329752 28383 leader_election.cc:343] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.329883 28383 leader_election.cc:304] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.330112 29643 tablet_replica.cc:333] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.330343 29643 raft_consensus.cc:2243] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.330693 29643 raft_consensus.cc:2272] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.330726 29998 raft_consensus.cc:2749] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.331311 29643 ts_tablet_manager.cc:1916] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.337219 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8a18f32c29224bd883ceb20658dff228 (table default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]) successfully deleted
I20260320 15:27:39.337309 28733 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.337505 28733 ts_tablet_manager.cc:1403] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.105s user 0.003s sys 0.000s
I20260320 15:27:39.338585 28733 raft_consensus.cc:359] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.338848 28733 raft_consensus.cc:385] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.338920 28733 raft_consensus.cc:740] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.339180 28733 consensus_queue.cc:260] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.340350 29676 raft_consensus.cc:493] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.340543 29676 raft_consensus.cc:515] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:39.340602 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 94) took 2761 ms (client timeout 1000 ms). Trace:
I20260320 15:27:39.340751 29998 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.340929 29998 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.341298 29676 leader_election.cc:290] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.341653 29998 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.342371 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.342571 28441 raft_consensus.cc:2468] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:39.342839 28382 leader_election.cc:343] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 757b5afdea674217909610e3719a8804
I20260320 15:27:39.342986 28524 leader_election.cc:304] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.343258 29676 raft_consensus.cc:2804] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:39.343338 29676 raft_consensus.cc:493] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:39.343411 29676 raft_consensus.cc:3060] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.347029 29676 raft_consensus.cc:515] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:39.346076 28905 rpcz_store.cc:269] 0320 15:27:36.578709 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.982506 (+403797us) service_pool.cc:225] Handling call
0320 15:27:39.340544 (+2358038us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:38.065107 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.065333 (+ 226us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.065344 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:38.065350 (+ 6us) tablet.cc:661] Decoding operations
0320 15:27:38.065544 (+ 194us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.065568 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.065576 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:38.065675 (+ 99us) tablet.cc:700] Row locks acquired
0320 15:27:38.065680 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:38.065734 (+ 54us) write_op.cc:270] Start()
0320 15:27:38.065788 (+ 54us) write_op.cc:276] Timestamp: P: 1774020458065649 usec, L: 0
0320 15:27:38.065794 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.066088 (+ 294us) log.cc:844] Serialized 527 byte log entry
0320 15:27:38.113954 (+ 47866us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.116729 (+ 2775us) write_op.cc:301] APPLY: starting
0320 15:27:38.116753 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.116806 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.116838 (+ 32us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.116911 (+ 73us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.116952 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:38.116960 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:38.117089 (+ 129us) log.cc:844] Serialized 23 byte log entry
0320 15:27:38.117170 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.117195 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:38.117214 (+ 19us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:38.303346 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.303562 (+ 216us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.303608 (+ 46us) write_op.cc:435] Acquired schema lock
0320 15:27:38.303613 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:38.303765 (+ 152us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.303787 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.303796 (+ 9us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:38.303888 (+ 92us) tablet.cc:700] Row locks acquired
0320 15:27:38.303892 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:38.303946 (+ 54us) write_op.cc:270] Start()
0320 15:27:38.303994 (+ 48us) write_op.cc:276] Timestamp: P: 1774020458303904 usec, L: 0
0320 15:27:38.304000 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.304273 (+ 273us) log.cc:844] Serialized 356 byte log entry
0320 15:27:38.310356 (+ 6083us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.310503 (+ 147us) write_op.cc:301] APPLY: starting
0320 15:27:38.310526 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.310590 (+ 64us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.310634 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.310698 (+ 64us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.310735 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:38.310742 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:38.310829 (+ 87us) log.cc:844] Serialized 23 byte log entry
0320 15:27:38.310898 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.310922 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:38.310930 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:38.311004 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.311008 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:39.021046 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.021282 (+ 236us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.021293 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:39.021297 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:39.021455 (+ 158us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.021479 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.021486 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.021578 (+ 92us) tablet.cc:700] Row locks acquired
0320 15:27:39.021581 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:39.021641 (+ 60us) write_op.cc:270] Start()
0320 15:27:39.021692 (+ 51us) write_op.cc:276] Timestamp: P: 1774020459021565 usec, L: 0
0320 15:27:39.021697 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.021983 (+ 286us) log.cc:844] Serialized 356 byte log entry
0320 15:27:39.028942 (+ 6959us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.029064 (+ 122us) write_op.cc:301] APPLY: starting
0320 15:27:39.029086 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.029135 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.029178 (+ 43us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.029260 (+ 82us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.029298 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.029306 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.029403 (+ 97us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.029475 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.029501 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:39.029509 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.029588 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.029592 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":131690,"reported_tablets":221,"child_traces":[["op",{"apply.queue_time_us":2660,"apply.run_cpu_time_us":580,"apply.run_wall_time_us":580,"num_ops":1,"prepare.queue_time_us":21207,"prepare.run_cpu_time_us":1183,"prepare.run_wall_time_us":1184,"raft.queue_time_us":914,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":451,"replication_time_us":48109,"thread_start_us":36206,"threads_started":2}],["op",{"apply.queue_time_us":58,"apply.run_cpu_time_us":504,"apply.run_wall_time_us":503,"num_ops":1,"prepare.queue_time_us":68,"prepare.run_cpu_time_us":1035,"prepare.run_wall_time_us":1080,"raft.queue_time_us":583,"raft.run_cpu_time_us":530,"raft.run_wall_time_us":530,"replication_time_us":6305}],["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":514,"num_ops":1,"prepare.queue_time_us":8902,"prepare.run_cpu_time_us":1121,"prepare.run_wall_time_us":1120,"raft.queue_time_us":578,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":371,"replication_time_us":7205}]]}
I20260320 15:27:39.348174 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e708aa24a35c49d88ac8f3e84bcdf4e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:39.353827 29676 leader_election.cc:290] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.356101 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:39.356343 28441 raft_consensus.cc:3060] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.357363 28736 tablet_bootstrap.cc:654] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.359086 28441 raft_consensus.cc:2468] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:39.359723 28524 leader_election.cc:304] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.360091 29676 raft_consensus.cc:2804] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
W20260320 15:27:39.360216 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e708aa24a35c49d88ac8f3e84bcdf4e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.360275 29676 raft_consensus.cc:697] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:39.360671 29676 consensus_queue.cc:237] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.366274 29599 consensus_queue.cc:1048] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.369091 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 114) took 5236 ms. Trace:
I20260320 15:27:39.369233 29678 rpcz_store.cc:276] 0320 15:27:34.132380 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.132467 (+ 87us) service_pool.cc:225] Handling call
0320 15:27:39.238807 (+5106340us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:39.238825 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.315072 (+ 76247us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.319275 (+ 4203us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.369068 (+ 49793us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5092428}
I20260320 15:27:39.369635 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 118) took 5223 ms. Trace:
I20260320 15:27:39.369705 29678 rpcz_store.cc:276] 0320 15:27:34.146446 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.146519 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:39.369529 (+5223010us) ts_tablet_manager.cc:1151] Deleting tablet 825f7e3fdc004fe1a3bcca6deb20cccd
0320 15:27:39.369542 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.369618 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5222735}
I20260320 15:27:39.370052 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 115) took 5233 ms. Trace:
I20260320 15:27:39.370122 29678 rpcz_store.cc:276] 0320 15:27:34.136805 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.136951 (+ 146us) service_pool.cc:225] Handling call
0320 15:27:39.369927 (+5232976us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:39.369935 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.370037 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5222805}
I20260320 15:27:39.370371 29678 tablet_replica.cc:333] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:39.370607 29678 raft_consensus.cc:2243] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.370780 29678 raft_consensus.cc:2272] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.371433 29678 ts_tablet_manager.cc:1916] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.379345 29678 ts_tablet_manager.cc:1929] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.379895 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.380460 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.381063 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet edb7b53bb7444a39b471218c6039542f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.381660 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.382964 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.383803 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 993e2fb1561f4317be85e9b5e5816edc with delete_type TABLET_DATA_DELETED (replaced by tablet 583ed053b2a742c7b029f348f4495101) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.384654 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.384786 29643 ts_tablet_manager.cc:1929] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.384938 29643 log.cc:1199] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:39.384261 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:39.385327 29643 ts_tablet_manager.cc:1950] T f5f92561764b4cb9b15bb9a310a350f0 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.385691 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 106) took 5302 ms. Trace:
I20260320 15:27:39.385790 29643 rpcz_store.cc:276] 0320 15:27:34.082750 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.082806 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:39.326810 (+5244004us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:39.330025 (+ 3215us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.334405 (+ 4380us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.384644 (+ 50239us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.385659 (+ 1015us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5243799}
I20260320 15:27:39.386359 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet f5f92561764b4cb9b15bb9a310a350f0 (table default.Table_3a07fada18374bf297e97be27789813f [id=3ac0cb8680a34e1087cf65c8986780a7]) successfully deleted
I20260320 15:27:39.388135 29599 consensus_queue.cc:1048] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.389364 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:39.391381 29998 raft_consensus.cc:493] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.391608 29998 raft_consensus.cc:515] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.391520 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:39.392344 29998 leader_election.cc:290] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.392786 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:39.393441 28382 leader_election.cc:343] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.393052 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:39.395893 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet adb65832eefe4bd4b08d95aa83c7bd7b (table default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]) successfully deleted
W20260320 15:27:39.396296 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 825f7e3fdc004fe1a3bcca6deb20cccd because the tablet was not found. No further retry: Not found: Tablet not found: 825f7e3fdc004fe1a3bcca6deb20cccd
W20260320 15:27:39.396689 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 71ec63fb080545de838803033f204fab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 71ec63fb080545de838803033f204fab already in progress: creating tablet
I20260320 15:27:39.403415 28733 ts_tablet_manager.cc:1434] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.066s user 0.002s sys 0.002s
I20260320 15:27:39.403833 28733 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:39.405678 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:39.406888 28733 tablet_bootstrap.cc:654] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.407369 28903 catalog_manager.cc:5671] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:39.410742 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:39.415056 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a9ddea49e641da881fba339ddef549 with delete_type TABLET_DATA_DELETED (replaced by tablet 4becc525b59d421caf79d8cd42052118) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.414551 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:39.419701 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:39.423763 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet c752fb99ef494bda9d144d0e180e3100 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.422602 29643 tablet_replica.cc:333] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.427407 29643 raft_consensus.cc:2243] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.426362 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:39.430858 29643 raft_consensus.cc:2272] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.431609 29643 ts_tablet_manager.cc:1916] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.432376 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:39.433553 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 74) took 5430 ms. Trace:
I20260320 15:27:39.433698 28892 rpcz_store.cc:276] 0320 15:27:34.003350 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.181431 (+5178081us) service_pool.cc:225] Handling call
0320 15:27:39.393931 (+212500us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.394763 (+ 832us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.402909 (+ 8146us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.433355 (+ 30446us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.433491 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.396591 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.396743 (+ 152us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.396751 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:39.396753 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.396858 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.396875 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.396879 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.396977 (+ 98us) tablet.cc:700] Row locks acquired
0320 15:27:39.396981 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:39.397033 (+ 52us) write_op.cc:270] Start()
0320 15:27:39.397078 (+ 45us) write_op.cc:276] Timestamp: P: 1774020459396980 usec, L: 0
0320 15:27:39.397081 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.397353 (+ 272us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.402185 (+ 4832us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.402319 (+ 134us) write_op.cc:301] APPLY: starting
0320 15:27:39.402339 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.402408 (+ 69us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.402445 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.402565 (+ 120us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.402603 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.402611 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.402697 (+ 86us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.402770 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.402793 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:39.402799 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.402886 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.402890 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":555,"apply.run_wall_time_us":554,"num_ops":2,"prepare.queue_time_us":1197,"prepare.run_cpu_time_us":890,"prepare.run_wall_time_us":890,"raft.queue_time_us":386,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":276,"replication_time_us":5068}]]}
I20260320 15:27:39.434672 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e0d5290fa4ae42909d4be711b08b4018"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.435442 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0d5290fa4ae42909d4be711b08b4018 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.436265 29617 consensus_queue.cc:1048] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.436877 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 75) took 5433 ms. Trace:
I20260320 15:27:39.437004 28905 rpcz_store.cc:276] 0320 15:27:34.003513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.347916 (+5344403us) service_pool.cc:225] Handling call
0320 15:27:39.395388 (+ 47472us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.395957 (+ 569us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.408696 (+ 12739us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.436728 (+ 28032us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.436833 (+ 105us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.397909 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.398058 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.398065 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:39.398068 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:39.398169 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.398186 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.398194 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.398377 (+ 183us) tablet.cc:700] Row locks acquired
0320 15:27:39.398380 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:39.398427 (+ 47us) write_op.cc:270] Start()
0320 15:27:39.398469 (+ 42us) write_op.cc:276] Timestamp: P: 1774020459398377 usec, L: 0
0320 15:27:39.398475 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.398668 (+ 193us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.407977 (+ 9309us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.408085 (+ 108us) write_op.cc:301] APPLY: starting
0320 15:27:39.408103 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.408151 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.408189 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.408320 (+ 131us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.408388 (+ 68us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.408397 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:39.408490 (+ 93us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.408559 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.408586 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:39.408593 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.408667 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.408671 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":60,"apply.run_cpu_time_us":569,"apply.run_wall_time_us":568,"mutex_wait_us":1313,"num_ops":2,"prepare.queue_time_us":1377,"prepare.run_cpu_time_us":856,"prepare.run_wall_time_us":855,"replication_time_us":9465}]]}
I20260320 15:27:39.437888 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_cfd32ab97f564c6dbe89916a2960fe1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.438617 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfd32ab97f564c6dbe89916a2960fe1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.447731 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 76) took 5444 ms. Trace:
I20260320 15:27:39.447907 28892 rpcz_store.cc:276] 0320 15:27:34.003681 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.434407 (+5430726us) service_pool.cc:225] Handling call
0320 15:27:39.435599 (+ 1192us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.436120 (+ 521us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.447402 (+ 11282us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.447576 (+ 174us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.447683 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.441194 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.441338 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.441345 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:39.441347 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.441446 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.441462 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.441467 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.441562 (+ 95us) tablet.cc:700] Row locks acquired
0320 15:27:39.441564 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:39.441609 (+ 45us) write_op.cc:270] Start()
0320 15:27:39.441651 (+ 42us) write_op.cc:276] Timestamp: P: 1774020459441557 usec, L: 0
0320 15:27:39.441656 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.441938 (+ 282us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.446662 (+ 4724us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.446780 (+ 118us) write_op.cc:301] APPLY: starting
0320 15:27:39.446800 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.446849 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.446889 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.447056 (+ 167us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.447093 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.447101 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.447191 (+ 90us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.447263 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.447286 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:39.447293 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.447376 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.447379 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":68,"apply.run_cpu_time_us":585,"apply.run_wall_time_us":585,"num_ops":2,"prepare.queue_time_us":4484,"prepare.run_cpu_time_us":883,"prepare.run_wall_time_us":883,"raft.queue_time_us":406,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":277,"replication_time_us":4969}]]}
I20260320 15:27:39.448786 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_3f812789f8e54c0ab7870ab350295947"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.449491 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f812789f8e54c0ab7870ab350295947 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.458128 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 77) took 5401 ms. Trace:
I20260320 15:27:39.458314 28905 rpcz_store.cc:276] 0320 15:27:34.057150 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.437610 (+5380460us) service_pool.cc:225] Handling call
0320 15:27:39.438762 (+ 1152us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.439386 (+ 624us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.457675 (+ 18289us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.457839 (+ 164us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.458066 (+ 227us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.442210 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.442390 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.442400 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:39.442404 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:39.442532 (+ 128us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.442547 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.442553 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.442672 (+ 119us) tablet.cc:700] Row locks acquired
0320 15:27:39.442677 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:39.442717 (+ 40us) write_op.cc:270] Start()
0320 15:27:39.442753 (+ 36us) write_op.cc:276] Timestamp: P: 1774020459442667 usec, L: 0
0320 15:27:39.442756 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.442955 (+ 199us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.456657 (+ 13702us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.456959 (+ 302us) write_op.cc:301] APPLY: starting
0320 15:27:39.456981 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.457036 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.457069 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.457221 (+ 152us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.457263 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.457271 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.457374 (+ 103us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.457448 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.457542 (+ 94us) write_op.cc:454] Released schema lock
0320 15:27:39.457550 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.457646 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.457650 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":678,"apply.run_wall_time_us":678,"mutex_wait_us":1400,"num_ops":2,"prepare.queue_time_us":1450,"prepare.run_cpu_time_us":828,"prepare.run_wall_time_us":827,"replication_time_us":13684}]]}
I20260320 15:27:39.459239 28905 master_service.cc:954] ReplaceTablet: received request to replace tablet 7c4b53fb135741efad5d3e59a6d4eb3e from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:39.460391 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:39.461495 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:39.462564 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:39.462884 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:39.463332 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:39.463948 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:39.464262 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:39.464646 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:39.465178 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 78) took 5407 ms. Trace:
I20260320 15:27:39.465384 28892 rpcz_store.cc:276] 0320 15:27:34.057476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.448564 (+5391088us) service_pool.cc:225] Handling call
0320 15:27:39.449632 (+ 1068us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.450186 (+ 554us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.464893 (+ 14707us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.465012 (+ 119us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.465124 (+ 112us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.451428 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.451599 (+ 171us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.451606 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:39.451608 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.451706 (+ 98us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.451722 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.451726 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.451823 (+ 97us) tablet.cc:700] Row locks acquired
0320 15:27:39.451828 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:39.451868 (+ 40us) write_op.cc:270] Start()
0320 15:27:39.451907 (+ 39us) write_op.cc:276] Timestamp: P: 1774020459451816 usec, L: 0
0320 15:27:39.451910 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.452155 (+ 245us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.463630 (+ 11475us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.464196 (+ 566us) write_op.cc:301] APPLY: starting
0320 15:27:39.464217 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.464274 (+ 57us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.464320 (+ 46us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.464460 (+ 140us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.464530 (+ 70us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.464541 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:39.464643 (+ 102us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.464726 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.464752 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:39.464760 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.464840 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.464844 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":481,"apply.run_cpu_time_us":646,"apply.run_wall_time_us":647,"num_ops":2,"prepare.queue_time_us":670,"prepare.run_cpu_time_us":826,"prepare.run_wall_time_us":825,"replication_time_us":11683}]]}
I20260320 15:27:39.466650 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_39acdde6b06e4e4e9a09138804663dba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:39.466991 28734 tablet_bootstrap.cc:654] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:39.467891 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39acdde6b06e4e4e9a09138804663dba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.470805 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:39.381477 30010 raft_consensus.cc:3060] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.394749 30114 raft_consensus.cc:493] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.395571 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.396027 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.396361 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 4becc525b59d421caf79d8cd42052118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.396703 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 6231fe30c5b34d0bb7a233abbffeb1e8 with delete_type TABLET_DATA_DELETED (replaced by tablet e779a274913f4caeaf82e3f180457830) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.397697 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.474196 30114 raft_consensus.cc:515] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.398324 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 22b7f757ebdf456eb81085e5be54da4b with delete_type TABLET_DATA_DELETED (replaced by tablet e2586d40343b4365ab8a67cce2e1c6ca) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.398996 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.401083 30018 raft_consensus.cc:493] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.403208 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.474642 28303 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:39.474620 30018 raft_consensus.cc:515] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.474972 30114 leader_election.cc:290] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.475190 30018 leader_election.cc:290] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:39.475330 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:39.475724 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:39.476152 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.476398 28446 raft_consensus.cc:2393] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:39.476359 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:39.476511 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:39.476646 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:39.477104 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:39.477377 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:39.477520 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:39.477746 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:39.477989 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:39.478263 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:39.478442 28903 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:39.478374 30010 raft_consensus.cc:515] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.479010 30010 leader_election.cc:290] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:39.479717 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.479993 28443 raft_consensus.cc:2393] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:39.480427 29678 log.cc:1199] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7c4b53fb135741efad5d3e59a6d4eb3e
I20260320 15:27:39.480911 29678 ts_tablet_manager.cc:1950] T 7c4b53fb135741efad5d3e59a6d4eb3e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:39.482872 28523 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.74.60:42563, user_credentials={real_user=slave}} blocked reactor thread for 54053.3us
I20260320 15:27:39.485916 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:39.486148 28443 raft_consensus.cc:3060] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.490011 28443 raft_consensus.cc:2468] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:39.490744 28240 leader_election.cc:304] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.491053 30010 raft_consensus.cc:2804] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:39.491428 30010 raft_consensus.cc:697] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:39.491909 30010 consensus_queue.cc:237] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.494078 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 80) took 5034 ms. Trace:
I20260320 15:27:39.494271 28892 rpcz_store.cc:276] 0320 15:27:34.460018 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.466390 (+5006372us) service_pool.cc:225] Handling call
0320 15:27:39.468099 (+ 1709us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.468693 (+ 594us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.493746 (+ 25053us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.493893 (+ 147us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.494016 (+ 123us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.469327 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.469454 (+ 127us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.469460 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:39.469462 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.469561 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.469578 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.469585 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.469673 (+ 88us) tablet.cc:700] Row locks acquired
0320 15:27:39.469678 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:39.469721 (+ 43us) write_op.cc:270] Start()
0320 15:27:39.469756 (+ 35us) write_op.cc:276] Timestamp: P: 1774020459469667 usec, L: 0
0320 15:27:39.469760 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.469950 (+ 190us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.478907 (+ 8957us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.493177 (+ 14270us) write_op.cc:301] APPLY: starting
0320 15:27:39.493195 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.493235 (+ 40us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.493283 (+ 48us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.493385 (+ 102us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.493417 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.493424 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:39.493504 (+ 80us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.493567 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.493615 (+ 48us) write_op.cc:454] Released schema lock
0320 15:27:39.493622 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.493738 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.493743 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14203,"apply.run_cpu_time_us":550,"apply.run_wall_time_us":550,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":736,"prepare.run_wall_time_us":736,"raft.queue_time_us":363,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":268,"replication_time_us":9106}]]}
I20260320 15:27:39.495335 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_8d69a7f0253d4f47a24fcabe3b0dff20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.496104 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8d69a7f0253d4f47a24fcabe3b0dff20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.467192 28736 tablet_bootstrap.cc:492] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:39.498430 28736 ts_tablet_manager.cc:1403] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.204s user 0.002s sys 0.002s
I20260320 15:27:39.481822 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.482044 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.482110 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffe278739bbc4ddab848af9c25241380" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:39.499289 28736 raft_consensus.cc:359] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.499517 28736 raft_consensus.cc:385] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.499596 28736 raft_consensus.cc:740] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.499806 28736 consensus_queue.cc:260] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.502585 28905 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 79) took 5258 ms. Trace:
I20260320 15:27:39.502760 28905 rpcz_store.cc:276] 0320 15:27:34.244545 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.459164 (+5214619us) service_pool.cc:225] Handling call
0320 15:27:39.502477 (+ 43313us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:39.503396 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_1125a332fd6d45dfbd5dfbd174357989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.504179 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1125a332fd6d45dfbd5dfbd174357989 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.509354 28383 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.509665 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 81) took 4950 ms. Trace:
I20260320 15:27:39.510037 29842 raft_consensus.cc:2804] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:39.510138 29842 raft_consensus.cc:493] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.510221 29842 raft_consensus.cc:3060] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:39.509805 28892 rpcz_store.cc:276] 0320 15:27:34.559070 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.495088 (+4936018us) service_pool.cc:225] Handling call
0320 15:27:39.496245 (+ 1157us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.496813 (+ 568us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.509370 (+ 12557us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.509498 (+ 128us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.509605 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.497984 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.498115 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.498124 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:39.498129 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:39.498244 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.498259 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.498266 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.498338 (+ 72us) tablet.cc:700] Row locks acquired
0320 15:27:39.498341 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:39.498378 (+ 37us) write_op.cc:270] Start()
0320 15:27:39.498410 (+ 32us) write_op.cc:276] Timestamp: P: 1774020459498323 usec, L: 0
0320 15:27:39.498413 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.498592 (+ 179us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.505745 (+ 7153us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.508421 (+ 2676us) write_op.cc:301] APPLY: starting
0320 15:27:39.508438 (+ 17us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.508483 (+ 45us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.508529 (+ 46us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.508625 (+ 96us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.508656 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.508662 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:39.508734 (+ 72us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.508807 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.508824 (+ 17us) write_op.cc:454] Released schema lock
0320 15:27:39.508830 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.508904 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.508908 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2606,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":464,"prepare.run_cpu_time_us":702,"prepare.run_wall_time_us":703,"replication_time_us":7301}]]}
I20260320 15:27:39.510931 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_1b61db70cc644eaeb675dfb95960f295"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:39.511716 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b61db70cc644eaeb675dfb95960f295 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:39.514328 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 82) took 4953 ms. Trace:
I20260320 15:27:39.514447 28905 rpcz_store.cc:276] 0320 15:27:34.561034 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.503076 (+4942042us) service_pool.cc:225] Handling call
0320 15:27:39.504313 (+ 1237us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.504924 (+ 611us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.514054 (+ 9130us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.514157 (+ 103us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.514248 (+ 91us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.505581 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.505731 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.505740 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:39.505745 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:39.505852 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.505868 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.505876 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.505979 (+ 103us) tablet.cc:700] Row locks acquired
0320 15:27:39.505983 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:39.506020 (+ 37us) write_op.cc:270] Start()
0320 15:27:39.506059 (+ 39us) write_op.cc:276] Timestamp: P: 1774020459505963 usec, L: 0
0320 15:27:39.506064 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.506283 (+ 219us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.513287 (+ 7004us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.513603 (+ 316us) write_op.cc:301] APPLY: starting
0320 15:27:39.513619 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.513665 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.513691 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.513792 (+ 101us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.513824 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.513830 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:39.513900 (+ 70us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.513949 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.513965 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:39.513970 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.514052 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.514055 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":260,"apply.run_cpu_time_us":444,"apply.run_wall_time_us":443,"num_ops":2,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":785,"replication_time_us":7195}]]}
I20260320 15:27:39.515283 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_f2d3223c396f47d2b377874a6300645d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:39.516012 29842 raft_consensus.cc:515] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:39.516148 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2d3223c396f47d2b377874a6300645d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:39.518227 28382 leader_election.cc:343] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 757b5afdea674217909610e3719a8804
I20260320 15:27:39.518586 29842 leader_election.cc:290] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.521559 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 83) took 4952 ms. Trace:
I20260320 15:27:39.521831 28892 rpcz_store.cc:276] 0320 15:27:34.569322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.510560 (+4941238us) service_pool.cc:225] Handling call
0320 15:27:39.511831 (+ 1271us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.512395 (+ 564us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.521241 (+ 8846us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.521383 (+ 142us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.521499 (+ 116us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.512933 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.513030 (+ 97us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.513037 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:39.513039 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.513114 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.513124 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.513128 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.513218 (+ 90us) tablet.cc:700] Row locks acquired
0320 15:27:39.513220 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:39.513251 (+ 31us) write_op.cc:270] Start()
0320 15:27:39.513293 (+ 42us) write_op.cc:276] Timestamp: P: 1774020459513196 usec, L: 0
0320 15:27:39.513297 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.513549 (+ 252us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.520219 (+ 6670us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.520465 (+ 246us) write_op.cc:301] APPLY: starting
0320 15:27:39.520489 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.520550 (+ 61us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.520597 (+ 47us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.520704 (+ 107us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.520738 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.520746 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.520826 (+ 80us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.520888 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.520906 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:39.520912 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.520977 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.520981 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":194,"apply.run_cpu_time_us":508,"apply.run_wall_time_us":508,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":688,"prepare.run_wall_time_us":688,"replication_time_us":6891,"spinlock_wait_cycles":82816}]]}
I20260320 15:27:39.535869 28892 master_service.cc:954] ReplaceTablet: received request to replace tablet 48c1002cbdd845e2b96dcb57c450cc6d from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:39.535488 29643 ts_tablet_manager.cc:1929] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.544919 29643 log.cc:1199] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:39.545332 29643 ts_tablet_manager.cc:1950] T a4bfb87d5d5741adb0ad2f2a88f4867f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.547348 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 107) took 5464 ms. Trace:
I20260320 15:27:39.547582 29643 rpcz_store.cc:276] 0320 15:27:34.083387 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.083456 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:39.422507 (+5339051us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:39.422521 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.532256 (+109735us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.535348 (+ 3092us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.547271 (+ 11923us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5338766}
I20260320 15:27:39.548233 29643 tablet_replica.cc:333] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.548257 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet a4bfb87d5d5741adb0ad2f2a88f4867f (table default.Table_d046770ad1554426ab6791919207cdcc [id=0d1bce42ec29458a95121f253235d22f]) successfully deleted
I20260320 15:27:39.548668 29643 raft_consensus.cc:2243] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.549470 29643 raft_consensus.cc:2272] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.550865 29643 ts_tablet_manager.cc:1916] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.562040 29643 ts_tablet_manager.cc:1929] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.562338 29643 log.cc:1199] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:39.563212 29643 ts_tablet_manager.cc:1950] T b2355eadcae6483bae7ae475d74dd131 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.569334 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 108) took 5485 ms. Trace:
I20260320 15:27:39.569602 29643 rpcz_store.cc:276] 0320 15:27:34.084094 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.084151 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:39.548102 (+5463951us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:39.548113 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.556433 (+ 8320us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.561751 (+ 5318us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.569226 (+ 7475us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5463422}
I20260320 15:27:39.570281 29643 tablet_replica.cc:333] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.570684 29643 raft_consensus.cc:2243] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.570925 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b2355eadcae6483bae7ae475d74dd131 (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:39.571326 29643 raft_consensus.cc:2272] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.579774 28734 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:39.580127 28734 ts_tablet_manager.cc:1403] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.323s user 0.007s sys 0.000s
I20260320 15:27:39.587103 29643 ts_tablet_manager.cc:1916] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.588169 28736 ts_tablet_manager.cc:1434] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.090s user 0.002s sys 0.001s
I20260320 15:27:39.588851 28736 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:39.589851 28733 tablet_bootstrap.cc:492] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.591336 28733 ts_tablet_manager.cc:1403] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.188s user 0.005s sys 0.000s
I20260320 15:27:39.603426 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 116) took 5457 ms. Trace:
I20260320 15:27:39.603120 28733 raft_consensus.cc:359] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.605010 28733 raft_consensus.cc:385] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.605914 28733 raft_consensus.cc:740] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.608340 28733 consensus_queue.cc:260] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.614686 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3bc4dd47817490d88725127b456cc70" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:39.615060 28733 ts_tablet_manager.cc:1434] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.023s user 0.004s sys 0.000s
I20260320 15:27:39.616492 29678 rpcz_store.cc:276] 0320 15:27:34.145536 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.145600 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:39.370333 (+5224733us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:39.370340 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.375869 (+ 5529us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.379222 (+ 3353us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.603394 (+224172us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":60427,"tablet-delete.queue_time_us":5221983}
I20260320 15:27:39.618424 28733 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:39.619867 28524 leader_election.cc:343] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.620467 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bf5e17dea5445d9909a229fa8315348" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:39.620332 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:39.621958 29678 tablet_replica.cc:333] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
W20260320 15:27:39.622550 28524 leader_election.cc:343] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.623134 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcff253b9cce45b4ad11dc38397afee2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:39.623340 28584 raft_consensus.cc:3060] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:39.623703 28383 leader_election.cc:343] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.623847 28383 leader_election.cc:304] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.624737 29842 raft_consensus.cc:2749] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.626791 28584 raft_consensus.cc:2468] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:39.627111 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:39.629837 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:39.634651 29640 ts_tablet_manager.cc:1929] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.635394 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:39.640038 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:39.642169 29678 raft_consensus.cc:2243] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.643649 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:39.645880 29643 ts_tablet_manager.cc:1929] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.646037 29643 log.cc:1199] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:39.646457 29643 ts_tablet_manager.cc:1950] T 26818e620ff649d18b671e4f4c7c7e74 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.646478 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:39.649091 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 84) took 5010 ms. Trace:
I20260320 15:27:39.649312 28905 rpcz_store.cc:276] 0320 15:27:34.638469 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.515052 (+4876583us) service_pool.cc:225] Handling call
0320 15:27:39.516289 (+ 1237us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:39.516790 (+ 501us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:39.529884 (+ 13094us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:39.648877 (+118993us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:39.649007 (+ 130us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.517364 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.517470 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.517476 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:39.517478 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.517563 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.517578 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.517585 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.517675 (+ 90us) tablet.cc:700] Row locks acquired
0320 15:27:39.517679 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:39.517720 (+ 41us) write_op.cc:270] Start()
0320 15:27:39.517750 (+ 30us) write_op.cc:276] Timestamp: P: 1774020459517663 usec, L: 0
0320 15:27:39.517754 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.517937 (+ 183us) log.cc:844] Serialized 433 byte log entry
0320 15:27:39.524917 (+ 6980us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.525483 (+ 566us) write_op.cc:301] APPLY: starting
0320 15:27:39.525573 (+ 90us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.525623 (+ 50us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.525669 (+ 46us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.525807 (+ 138us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.525843 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:39.525850 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:39.525933 (+ 83us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.525994 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.526015 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:39.526022 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.526090 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.526094 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":498,"apply.run_cpu_time_us":559,"apply.run_wall_time_us":608,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":648,"prepare.run_wall_time_us":649,"replication_time_us":7127}]]}
I20260320 15:27:39.650465 29678 raft_consensus.cc:2272] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.653093 29678 ts_tablet_manager.cc:1916] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.653595 29640 log.cc:1199] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/bcec6175c1ca49f5b682a29d5c1e235a
I20260320 15:27:39.655666 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 109) took 5566 ms. Trace:
I20260320 15:27:39.655810 29643 rpcz_store.cc:276] 0320 15:27:34.088941 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.089024 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:39.570150 (+5481126us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:39.570163 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.632273 (+ 62110us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.645737 (+ 13464us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.655610 (+ 9873us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5480813}
I20260320 15:27:39.656872 29640 ts_tablet_manager.cc:1950] T bcec6175c1ca49f5b682a29d5c1e235a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:39.656373 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 110) took 5551 ms. Trace:
I20260320 15:27:39.658762 29643 rpcz_store.cc:276] 0320 15:27:34.105388 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.105471 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:39.656209 (+5550738us) ts_tablet_manager.cc:1151] Deleting tablet 825f7e3fdc004fe1a3bcca6deb20cccd
0320 15:27:39.656221 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.656323 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5526406}
I20260320 15:27:39.659155 29643 tablet_replica.cc:333] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.659415 29643 raft_consensus.cc:2243] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.659883 29643 raft_consensus.cc:2272] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.659967 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 26818e620ff649d18b671e4f4c7c7e74 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
W20260320 15:27:39.660329 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 825f7e3fdc004fe1a3bcca6deb20cccd because the tablet was not found. No further retry: Not found: Tablet not found: 825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:39.660514 29643 ts_tablet_manager.cc:1916] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.662425 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:39.663326 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:39.672261 29643 ts_tablet_manager.cc:1929] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.672430 29643 log.cc:1199] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/89425da569794a8fa972817023bf853d
I20260320 15:27:39.672857 29643 ts_tablet_manager.cc:1950] T 89425da569794a8fa972817023bf853d P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.673722 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 111) took 5567 ms. Trace:
I20260320 15:27:39.673843 29643 rpcz_store.cc:276] 0320 15:27:34.106127 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.106189 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:39.659078 (+5552889us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:39.659090 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.669180 (+ 10090us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.672133 (+ 2953us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.673680 (+ 1547us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5529190}
I20260320 15:27:39.674245 29643 tablet_replica.cc:333] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.674475 29643 raft_consensus.cc:2243] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.674683 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 89425da569794a8fa972817023bf853d (table default.Table_9f87c6321cdc405ab18f525e320707f2 [id=c4badbffe34944afa17c48384084a51c]) successfully deleted
I20260320 15:27:39.675261 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:39.676121 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:39.676718 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:39.676921 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:39.677129 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:39.677520 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:39.677738 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:39.678189 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:39.678699 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:39.679378 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:39.674676 29643 raft_consensus.cc:2272] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.682070 29643 ts_tablet_manager.cc:1916] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.657433 28239 leader_election.cc:304] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:39.685539 30114 raft_consensus.cc:3060] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.689004 30114 raft_consensus.cc:2749] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.689935 28239 leader_election.cc:304] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:39.690407 28239 raft_consensus.cc:2686] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:39.692509 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.692884 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.693488 30114 raft_consensus.cc:493] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.693655 30114 raft_consensus.cc:515] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.694932 30114 leader_election.cc:290] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:39.695101 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.695379 28444 raft_consensus.cc:2393] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:39.697160 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.697482 30114 consensus_queue.cc:1048] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.701956 29643 ts_tablet_manager.cc:1929] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:39.702140 29643 log.cc:1199] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/df5ffaedeaf94def93d93a538c42e3bd
I20260320 15:27:39.702579 29643 ts_tablet_manager.cc:1950] T df5ffaedeaf94def93d93a538c42e3bd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.702965 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:39.703485 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 115) took 5510 ms. Trace:
I20260320 15:27:39.704325 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.704551 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 5bc3fb47136d4b88a2898af399d74a8a with delete_type TABLET_DATA_DELETED (replaced by tablet 35347116811e4aee8142a503dd21843a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.704726 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.705313 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.705561 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.703615 29643 rpcz_store.cc:276] 0320 15:27:34.193422 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.193502 (+ 80us) service_pool.cc:225] Handling call
0320 15:27:39.674164 (+5480662us) ts_tablet_manager.cc:1151] Deleting tablet df5ffaedeaf94def93d93a538c42e3bd
0320 15:27:39.674175 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.695728 (+ 21553us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.701735 (+ 6007us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.703431 (+ 1696us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5480421}
I20260320 15:27:39.706925 29643 tablet_replica.cc:333] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.707165 29643 raft_consensus.cc:2243] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:39.707327 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet df5ffaedeaf94def93d93a538c42e3bd (table default.Table_874d6a667bff4c3d815fcff90cae13f5 [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
I20260320 15:27:39.707617 29643 raft_consensus.cc:2272] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.708284 29643 ts_tablet_manager.cc:1916] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.710443 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:39.710780 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:39.713127 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:39.713601 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:39.713905 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:39.714318 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:39.714604 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:39.714527 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "757b5afdea674217909610e3719a8804" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:39.714759 28295 raft_consensus.cc:3060] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:39.714880 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:39.715284 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:39.718405 28295 raft_consensus.cc:2468] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 2.
I20260320 15:27:39.641933 29676 consensus_queue.cc:1048] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.638427 28733 tablet_bootstrap.cc:654] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.728160 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 108) took 5868 ms. Trace:
I20260320 15:27:39.728296 29640 rpcz_store.cc:276] 0320 15:27:33.859578 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.859720 (+ 142us) service_pool.cc:225] Handling call
0320 15:27:39.216552 (+5356832us) ts_tablet_manager.cc:1151] Deleting tablet bcec6175c1ca49f5b682a29d5c1e235a
0320 15:27:39.216565 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.410956 (+194391us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.634529 (+223573us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.728110 (+ 93581us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5346649}
I20260320 15:27:39.728706 29640 tablet_replica.cc:333] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:39.728965 29640 raft_consensus.cc:2243] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.648651 30137 consensus_queue.cc:1048] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260320 15:27:39.729189 29640 raft_consensus.cc:2272] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.675616 29939 consensus_queue.cc:1048] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.693154 29599 raft_consensus.cc:493] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.729801 29640 ts_tablet_manager.cc:1916] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.729933 29599 raft_consensus.cc:515] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.730742 29599 leader_election.cc:290] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.735976 28734 raft_consensus.cc:359] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.736200 28734 raft_consensus.cc:385] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.736274 28734 raft_consensus.cc:740] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.737947 28734 consensus_queue.cc:260] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.738544 28734 ts_tablet_manager.cc:1434] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.158s user 0.002s sys 0.000s
I20260320 15:27:39.738863 28734 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:39.739743 29618 consensus_queue.cc:1048] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.743314 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet bcec6175c1ca49f5b682a29d5c1e235a (table default.Table_d1cf5412b72c4a58931061b1a1aba1e6 [id=fc92586ccab74ae3960a1eacc8cbac42]) successfully deleted
I20260320 15:27:39.744716 28736 tablet_bootstrap.cc:654] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.746457 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.746701 28442 raft_consensus.cc:2468] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:39.727445 28890 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 115) took 5386 ms (client timeout 1000 ms). Trace:
W20260320 15:27:39.761377 28890 rpcz_store.cc:269] 0320 15:27:34.341212 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.350344 (+ 9132us) service_pool.cc:225] Handling call
0320 15:27:34.398310 (+ 47966us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:39.727380 (+5329070us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:35.626193 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:35.626350 (+ 157us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:35.626358 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:35.626360 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:35.626473 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:35.626488 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:35.626494 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:35.626570 (+ 76us) tablet.cc:700] Row locks acquired
0320 15:27:35.626573 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:35.626612 (+ 39us) write_op.cc:270] Start()
0320 15:27:35.626649 (+ 37us) write_op.cc:276] Timestamp: P: 1774020455626563 usec, L: 0
0320 15:27:35.626653 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:35.626873 (+ 220us) log.cc:844] Serialized 527 byte log entry
0320 15:27:35.630988 (+ 4115us) op_driver.cc:464] REPLICATION: finished
0320 15:27:35.631145 (+ 157us) write_op.cc:301] APPLY: starting
0320 15:27:35.631181 (+ 36us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:35.631237 (+ 56us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:35.631276 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:35.631347 (+ 71us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:35.631392 (+ 45us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:35.631402 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:35.631508 (+ 106us) log.cc:844] Serialized 23 byte log entry
0320 15:27:35.631614 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.631643 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:35.631652 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:35.631741 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:35.631748 (+ 7us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:37.065184 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:37.065337 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:37.065346 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:37.065351 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:37.065469 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:37.065548 (+ 79us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:37.065556 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:37.065684 (+ 128us) tablet.cc:700] Row locks acquired
0320 15:27:37.065686 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:37.065733 (+ 47us) write_op.cc:270] Start()
0320 15:27:37.065774 (+ 41us) write_op.cc:276] Timestamp: P: 1774020457065700 usec, L: 0
0320 15:27:37.065778 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:37.066024 (+ 246us) log.cc:844] Serialized 356 byte log entry
0320 15:27:37.071030 (+ 5006us) op_driver.cc:464] REPLICATION: finished
0320 15:27:37.071170 (+ 140us) write_op.cc:301] APPLY: starting
0320 15:27:37.071190 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:37.071237 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:37.071264 (+ 27us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:37.071325 (+ 61us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:37.071362 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:37.071372 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:37.071462 (+ 90us) log.cc:844] Serialized 23 byte log entry
0320 15:27:37.071559 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.071579 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:37.071586 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:37.071665 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.071669 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12640,"reported_tablets":208,"spinlock_wait_cycles":73813376,"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":583,"apply.run_wall_time_us":582,"num_ops":1,"prepare.queue_time_us":769,"prepare.run_cpu_time_us":816,"prepare.run_wall_time_us":830,"raft.queue_time_us":74,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":288,"replication_time_us":4288}],["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":488,"apply.run_wall_time_us":487,"num_ops":1,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":984,"prepare.run_wall_time_us":982,"raft.queue_time_us":1651,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":249,"replication_time_us":5214}]]}
I20260320 15:27:39.769054 28524 leader_election.cc:304] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.769596 29617 raft_consensus.cc:2804] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:39.769724 29617 raft_consensus.cc:493] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:39.769820 29617 raft_consensus.cc:3060] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.772614 29617 raft_consensus.cc:515] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.773353 29617 leader_election.cc:290] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.774121 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:39.774344 28444 raft_consensus.cc:3060] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.777509 28444 raft_consensus.cc:2468] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:39.778228 28524 leader_election.cc:304] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.778640 29599 raft_consensus.cc:2804] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:39.778833 29599 raft_consensus.cc:697] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:39.779165 29599 consensus_queue.cc:237] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.783048 28736 tablet_bootstrap.cc:492] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:39.783210 28736 ts_tablet_manager.cc:1403] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.195s user 0.004s sys 0.001s
I20260320 15:27:39.784089 28736 raft_consensus.cc:359] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.784278 28736 raft_consensus.cc:385] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.784330 28736 raft_consensus.cc:740] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.784716 28736 consensus_queue.cc:260] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.785089 28734 tablet_bootstrap.cc:654] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.785356 28736 ts_tablet_manager.cc:1434] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:39.785737 28736 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:39.786823 28890 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_6d5fc724ebb540cbb35c767edfe19833" } modify_external_catalogs: true
I20260320 15:27:39.788381 28890 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_6d5fc724ebb540cbb35c767edfe19833" } modify_external_catalogs: true
I20260320 15:27:39.784883 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:39.792560 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:39.793031 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:39.801071 28734 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:39.804679 28734 ts_tablet_manager.cc:1403] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.066s user 0.004s sys 0.000s
I20260320 15:27:39.819025 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 48c1002cbdd845e2b96dcb57c450cc6d
I20260320 15:27:39.822990 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:39.823045 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:39.813447 28734 raft_consensus.cc:359] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.823931 28734 raft_consensus.cc:385] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.825210 29640 ts_tablet_manager.cc:1929] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.825484 28734 raft_consensus.cc:740] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.827729 28734 consensus_queue.cc:260] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.829438 28736 tablet_bootstrap.cc:654] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.829669 29678 ts_tablet_manager.cc:1929] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:39.829948 29678 log.cc:1199] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:39.832777 29678 ts_tablet_manager.cc:1950] T 1bf5e17dea5445d9909a229fa8315348 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:39.834599 28734 ts_tablet_manager.cc:1434] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20260320 15:27:39.839627 28892 catalog_manager.cc:6741] ReplaceTablet: tablet 48c1002cbdd845e2b96dcb57c450cc6d deleted and replaced by tablet 740981a8159243088548b90b0c576028
I20260320 15:27:39.839875 28892 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 85) took 5019 ms. Trace:
I20260320 15:27:39.839985 28892 rpcz_store.cc:276] 0320 15:27:34.820314 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.535766 (+4715452us) service_pool.cc:225] Handling call
0320 15:27:39.839799 (+304033us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.812694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.812827 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.812836 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:39.812840 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:39.813013 (+ 173us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.813032 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.813040 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.813128 (+ 88us) tablet.cc:700] Row locks acquired
0320 15:27:39.813130 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:39.813172 (+ 42us) write_op.cc:270] Start()
0320 15:27:39.813240 (+ 68us) write_op.cc:276] Timestamp: P: 1774020459813133 usec, L: 0
0320 15:27:39.813245 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.813492 (+ 247us) log.cc:844] Serialized 622 byte log entry
0320 15:27:39.817396 (+ 3904us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.817514 (+ 118us) write_op.cc:301] APPLY: starting
0320 15:27:39.817533 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.817585 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.817612 (+ 27us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.817731 (+ 119us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.817802 (+ 71us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.817810 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.817894 (+ 84us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.817956 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.817978 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:39.817985 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.818091 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.818095 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19956,"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":542,"apply.run_wall_time_us":542,"num_ops":2,"prepare.queue_time_us":74,"prepare.run_cpu_time_us":928,"prepare.run_wall_time_us":928,"raft.queue_time_us":364,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":293,"replication_time_us":4116}]]}
I20260320 15:27:39.840945 28892 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d046770ad1554426ab6791919207cdcc" } new_table_name: "default.Table_910f9050e6d24365b23da47d9569910a" modify_external_catalogs: true
I20260320 15:27:39.843123 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7c4b53fb135741efad5d3e59a6d4eb3e (table default.Table_3796f2b714f64f18a0db9057941513ff [id=dede327e43e74b34b23d244805c6d1fe]) successfully deleted
I20260320 15:27:39.833019 28736 tablet_bootstrap.cc:492] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:39.834149 29939 consensus_queue.cc:1048] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.861671 28736 ts_tablet_manager.cc:1403] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.076s user 0.000s sys 0.006s
I20260320 15:27:39.834609 30137 consensus_queue.cc:1048] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.844691 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "533105ad6e054fe3a4db32855bc688bd" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:39.862556 28736 raft_consensus.cc:359] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.862722 28736 raft_consensus.cc:385] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.862772 28736 raft_consensus.cc:740] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.862972 28736 consensus_queue.cc:260] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.863356 28734 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:39.863363 28736 ts_tablet_manager.cc:1434] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:39.835071 29643 ts_tablet_manager.cc:1929] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.848659 28383 leader_election.cc:304] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:39.863682 29643 log.cc:1199] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7c4b53fb135741efad5d3e59a6d4eb3e
W20260320 15:27:39.864053 28383 consensus_peers.cc:597] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7394e941c99e4358ad7a33db84b15348. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.864115 29643 ts_tablet_manager.cc:1950] T 7c4b53fb135741efad5d3e59a6d4eb3e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:39.864564 29842 raft_consensus.cc:2804] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:39.865006 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 117) took 5671 ms. Trace:
I20260320 15:27:39.865115 29643 rpcz_store.cc:276] 0320 15:27:34.193673 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.194360 (+ 687us) service_pool.cc:225] Handling call
0320 15:27:39.706847 (+5512487us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:39.706859 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.767625 (+ 60766us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.822175 (+ 54550us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.864988 (+ 42813us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5512242}
I20260320 15:27:39.865586 29640 log.cc:1199] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:39.865736 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7c4b53fb135741efad5d3e59a6d4eb3e (table default.Table_3796f2b714f64f18a0db9057941513ff [id=dede327e43e74b34b23d244805c6d1fe]) successfully deleted
I20260320 15:27:39.861666 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 121) took 5714 ms. Trace:
I20260320 15:27:39.866325 29678 rpcz_store.cc:276] 0320 15:27:34.146831 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.147836 (+ 1005us) service_pool.cc:225] Handling call
0320 15:27:39.621890 (+5474054us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:39.621907 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.764474 (+142567us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.829484 (+ 65010us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.861650 (+ 32166us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":633,"tablet-delete.queue_time_us":5471676}
I20260320 15:27:39.866882 29678 tablet_replica.cc:333] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:39.868873 28734 tablet_bootstrap.cc:654] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.866080 29640 ts_tablet_manager.cc:1950] T bb7a910bd2904a40ae63608377206c7e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:39.876802 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 109) took 6008 ms. Trace:
I20260320 15:27:39.852754 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:39.852843 30018 consensus_queue.cc:1048] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.853371 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:39.860317 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.861073 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.846236 30010 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.877578 30010 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.878299 30010 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.872195 28736 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:39.875834 29618 consensus_queue.cc:1048] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.880306 28736 tablet_bootstrap.cc:654] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.890627 29678 raft_consensus.cc:2243] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:39.880296 28524 consensus_peers.cc:597] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.892324 29640 rpcz_store.cc:276] 0320 15:27:33.867866 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.867920 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:39.728620 (+5860700us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:39.728630 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.781431 (+ 52801us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.825068 (+ 43637us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.876734 (+ 51666us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5848962}
I20260320 15:27:39.893013 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet bb7a910bd2904a40ae63608377206c7e (table default.Table_abe934610fd24a898915bc1d3f4b3307 [id=abb88155f1c74050840f50bcf1c2e6dc]) successfully deleted
I20260320 15:27:39.893934 29678 raft_consensus.cc:2272] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.898419 28733 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.898619 28733 ts_tablet_manager.cc:1403] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.281s user 0.016s sys 0.026s
I20260320 15:27:39.899870 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:39.900405 28736 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:39.900579 28736 ts_tablet_manager.cc:1403] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
I20260320 15:27:39.900720 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:39.901289 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:39.901515 29640 tablet_replica.cc:333] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:39.901392 28736 raft_consensus.cc:359] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.901589 28736 raft_consensus.cc:385] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.901648 28736 raft_consensus.cc:740] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.901810 28733 raft_consensus.cc:359] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.901855 28736 consensus_queue.cc:260] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.902086 29678 ts_tablet_manager.cc:1916] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.902590 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.903010 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:39.903348 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:39.904028 28524 consensus_peers.cc:597] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.902081 28733 raft_consensus.cc:385] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.905494 28733 raft_consensus.cc:740] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.906190 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:39.906572 28733 consensus_queue.cc:260] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.908672 28733 ts_tablet_manager.cc:1434] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:39.909175 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:39.909884 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:39.911975 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:39.912985 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:39.913715 29640 raft_consensus.cc:2243] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:39.913868 28524 consensus_peers.cc:597] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:39.912029 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1bf5e17dea5445d9909a229fa8315348 (table default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]) successfully deleted
I20260320 15:27:39.878127 29643 tablet_replica.cc:333] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:39.879022 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.915155 29643 raft_consensus.cc:2243] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:39.915252 28442 raft_consensus.cc:2468] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:39.915341 29643 raft_consensus.cc:2272] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.915477 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:39.915967 29643 ts_tablet_manager.cc:1916] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:39.915889 28524 leader_election.cc:343] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.899554 29842 raft_consensus.cc:697] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:39.905167 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:39.916342 28524 leader_election.cc:343] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:39.916407 29842 consensus_queue.cc:237] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.916693 28240 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.916961 30010 raft_consensus.cc:2804] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:39.917047 30010 raft_consensus.cc:493] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:39.917122 30010 raft_consensus.cc:3060] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:39.919802 30010 raft_consensus.cc:515] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.920445 30010 leader_election.cc:290] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:39.921185 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc27a4968ca4203b67dd3064fb03868" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:39.921394 28442 raft_consensus.cc:3060] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:39.921909 28733 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:39.922195 28584 raft_consensus.cc:3060] T f05205970cd249b68d138b904ba61c9e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.922996 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:39.913987 29640 raft_consensus.cc:2272] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:39.924341 28442 raft_consensus.cc:2468] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 2.
I20260320 15:27:39.924932 29640 ts_tablet_manager.cc:1916] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:39.924973 28240 leader_election.cc:304] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:39.925532 30010 raft_consensus.cc:2804] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:39.925697 30010 raft_consensus.cc:697] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:39.925989 30010 consensus_queue.cc:237] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.927983 30114 consensus_queue.cc:1048] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.928186 28733 tablet_bootstrap.cc:654] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.929437 28736 ts_tablet_manager.cc:1434] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.029s user 0.001s sys 0.000s
I20260320 15:27:39.919867 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:39.930222 30114 consensus_queue.cc:1048] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.931473 30114 consensus_queue.cc:1048] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.931824 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:39.932224 28239 leader_election.cc:304] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:39.932499 30114 raft_consensus.cc:3060] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:39.932978 28904 catalog_manager.cc:5671] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:39.933508 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:39.934477 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:39.934713 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:39.934976 30114 raft_consensus.cc:2749] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:39.935127 29640 ts_tablet_manager.cc:1929] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:39.935287 29640 log.cc:1199] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/f70530cd63a0420fb51527759c951e31
W20260320 15:27:39.935925 28239 leader_election.cc:343] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:39.937978 30114 consensus_queue.cc:1048] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.949798 30153 consensus_queue.cc:1048] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:39.934978 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:39.953544 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:39.953795 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:39.954044 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:39.959203 28733 tablet_bootstrap.cc:492] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.959427 28733 ts_tablet_manager.cc:1403] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.001s
I20260320 15:27:39.960366 28733 raft_consensus.cc:359] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.960582 28733 raft_consensus.cc:385] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.960666 28733 raft_consensus.cc:740] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.961058 28733 consensus_queue.cc:260] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.961797 28733 ts_tablet_manager.cc:1434] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:39.962141 28733 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:39.964455 28734 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:39.964599 28734 ts_tablet_manager.cc:1403] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.000s
I20260320 15:27:39.965559 28734 raft_consensus.cc:359] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.965747 28734 raft_consensus.cc:385] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:39.965801 28734 raft_consensus.cc:740] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:39.966023 28734 consensus_queue.cc:260] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:39.966729 28734 ts_tablet_manager.cc:1434] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:39.967300 28734 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:39.969092 28734 tablet_bootstrap.cc:654] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.969180 28733 tablet_bootstrap.cc:654] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.967514 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:39.971239 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:39.972100 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:39.972325 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:39.977341 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:39.978106 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:39.978281 28736 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:39.978619 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:39.978811 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:39.979084 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:39.979481 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:39.979740 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:39.980311 28736 tablet_bootstrap.cc:654] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:39.979972 28904 catalog_manager.cc:5671] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:39.980836 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:39.984364 30155 raft_consensus.cc:493] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:39.984681 30155 raft_consensus.cc:515] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:39.985615 30155 leader_election.cc:290] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:39.986379 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:39.986655 28443 raft_consensus.cc:2393] T 685e581b3f084d4a9aa97fba0a7767c7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:39.981220 28904 catalog_manager.cc:5671] T a17aa9eba95d463b9c0fe5ae7e3cfe13 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:39.989002 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
W20260320 15:27:39.998775 28239 leader_election.cc:343] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:40.001606 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685e581b3f084d4a9aa97fba0a7767c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.008404 29643 ts_tablet_manager.cc:1929] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.008582 29643 log.cc:1199] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/583ed053b2a742c7b029f348f4495101
I20260320 15:27:40.009048 29643 ts_tablet_manager.cc:1950] T 583ed053b2a742c7b029f348f4495101 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.009482 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 120) took 5815 ms. Trace:
I20260320 15:27:40.009588 29643 rpcz_store.cc:276] 0320 15:27:34.194112 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.194473 (+ 361us) service_pool.cc:225] Handling call
0320 15:27:39.878074 (+5683601us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:39.878086 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.989603 (+111517us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.008258 (+ 18655us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.009461 (+ 1203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5683416}
I20260320 15:27:40.010150 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 583ed053b2a742c7b029f348f4495101 (table default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]) successfully deleted
I20260320 15:27:39.988428 28733 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:39.989753 30153 consensus_queue.cc:1048] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.022482 28733 ts_tablet_manager.cc:1403] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.002s
I20260320 15:27:39.989871 29939 consensus_queue.cc:1048] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.022918 29678 ts_tablet_manager.cc:1929] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.023072 29678 log.cc:1199] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:40.023435 28733 raft_consensus.cc:359] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.023823 28733 raft_consensus.cc:385] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.023919 28733 raft_consensus.cc:740] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.024147 28733 consensus_queue.cc:260] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.024991 28733 ts_tablet_manager.cc:1434] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:40.025355 28733 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.028030 28524 leader_election.cc:304] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:40.028630 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet db16d86664794305bde714a7ff4d5caf consensus state has no leader
I20260320 15:27:40.028875 30155 raft_consensus.cc:3060] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.031288 28733 tablet_bootstrap.cc:654] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.031308 30153 consensus_queue.cc:1048] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260320 15:27:40.044845 28736 tablet_bootstrap.cc:492] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.045058 28736 ts_tablet_manager.cc:1403] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.002s
I20260320 15:27:40.046252 28736 raft_consensus.cc:359] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.046579 28736 raft_consensus.cc:385] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.046784 28736 raft_consensus.cc:740] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.047127 28736 consensus_queue.cc:260] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.048036 28736 ts_tablet_manager.cc:1434] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:40.050026 29678 ts_tablet_manager.cc:1950] T 1c2d7e4d26994a17bf2d2615d6552412 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.054919 30018 consensus_queue.cc:1048] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.056437 29640 ts_tablet_manager.cc:1950] T f70530cd63a0420fb51527759c951e31 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:40.056926 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 110) took 6176 ms. Trace:
I20260320 15:27:40.061764 29640 rpcz_store.cc:276] 0320 15:27:33.880178 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.880232 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:39.901410 (+6021178us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:39.901423 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.931926 (+ 30503us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:39.934958 (+ 3032us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.056878 (+121920us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11878,"tablet-delete.queue_time_us":6000473}
I20260320 15:27:40.062551 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet f70530cd63a0420fb51527759c951e31 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:40.063023 29640 tablet_replica.cc:333] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:40.063313 29640 raft_consensus.cc:2243] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.063622 29640 raft_consensus.cc:2272] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.064473 29640 ts_tablet_manager.cc:1916] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.064889 28736 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.066939 30168 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:40.067200 30168 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.067417 29599 raft_consensus.cc:493] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.067677 29599 raft_consensus.cc:515] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.068823 29599 leader_election.cc:290] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.068800 30168 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.069175 29643 tablet_replica.cc:333] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.069414 29643 raft_consensus.cc:2243] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:40.069484 28382 leader_election.cc:343] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:40.069659 29643 raft_consensus.cc:2272] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.070554 29643 ts_tablet_manager.cc:1916] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.071352 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.071684 28445 raft_consensus.cc:2468] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:40.059365 30155 raft_consensus.cc:2749] T 685e581b3f084d4a9aa97fba0a7767c7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.076186 28736 tablet_bootstrap.cc:654] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.077987 28524 leader_election.cc:304] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.077386 28734 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.078368 28734 ts_tablet_manager.cc:1403] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.111s user 0.004s sys 0.000s
I20260320 15:27:40.079197 28734 raft_consensus.cc:359] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.079386 28734 raft_consensus.cc:385] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.079520 28734 raft_consensus.cc:740] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.079877 28734 consensus_queue.cc:260] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.080575 28734 ts_tablet_manager.cc:1434] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:40.080865 28734 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.082052 29599 raft_consensus.cc:2804] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.082242 29599 raft_consensus.cc:493] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.082389 29599 raft_consensus.cc:3060] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.082469 28734 tablet_bootstrap.cc:654] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.084995 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 122) took 5937 ms. Trace:
I20260320 15:27:40.085198 29678 rpcz_store.cc:276] 0320 15:27:34.147464 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.147637 (+ 173us) service_pool.cc:225] Handling call
0320 15:27:39.866814 (+5719177us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:39.866830 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:39.905388 (+ 38558us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.022783 (+117395us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.084948 (+ 62165us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12697,"tablet-delete.queue_time_us":5714946}
I20260320 15:27:40.085511 29599 raft_consensus.cc:515] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.086002 29678 tablet_replica.cc:333] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.086182 29599 leader_election.cc:290] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.086289 29678 raft_consensus.cc:2243] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.086514 29678 raft_consensus.cc:2272] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.087113 29678 ts_tablet_manager.cc:1916] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.092360 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.092705 28445 raft_consensus.cc:3060] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.093782 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.099352 30168 raft_consensus.cc:493] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.099607 30168 raft_consensus.cc:515] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.100308 30168 leader_election.cc:290] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.100880 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:40.101475 28382 leader_election.cc:343] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.101624 30137 consensus_queue.cc:1048] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.103317 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1c2d7e4d26994a17bf2d2615d6552412 (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]) successfully deleted
I20260320 15:27:40.104817 28445 raft_consensus.cc:2468] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.105491 28524 leader_election.cc:304] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.105901 30155 raft_consensus.cc:2804] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.106112 30155 raft_consensus.cc:697] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:40.106432 30155 consensus_queue.cc:237] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.107369 28736 tablet_bootstrap.cc:492] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.107568 28736 ts_tablet_manager.cc:1403] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.000s
I20260320 15:27:40.108399 28736 raft_consensus.cc:359] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.108584 28736 raft_consensus.cc:385] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.108656 28736 raft_consensus.cc:740] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.108897 28736 consensus_queue.cc:260] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.111990 30010 raft_consensus.cc:493] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.112149 28736 ts_tablet_manager.cc:1434] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:40.112172 30010 raft_consensus.cc:515] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.112514 28736 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.112910 30010 leader_election.cc:290] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.113541 29618 raft_consensus.cc:493] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.113688 29618 raft_consensus.cc:515] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.113703 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.114058 28445 raft_consensus.cc:2393] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:40.114638 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.114946 28442 raft_consensus.cc:2468] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:40.114902 29618 leader_election.cc:290] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.115657 28240 leader_election.cc:304] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.115911 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.116147 29618 raft_consensus.cc:2804] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.116243 29618 raft_consensus.cc:493] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260320 15:27:40.110870 28524 leader_election.cc:343] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.116657 29618 raft_consensus.cc:3060] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.118156 28736 tablet_bootstrap.cc:654] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.119287 28734 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.120059 29618 raft_consensus.cc:515] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.121323 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.121619 28442 raft_consensus.cc:3060] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.124310 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.124719 28442 raft_consensus.cc:2468] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:40.125259 28240 leader_election.cc:304] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.125759 30010 raft_consensus.cc:2804] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.125901 30010 raft_consensus.cc:697] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:40.126145 30010 consensus_queue.cc:237] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.126278 29618 leader_election.cc:290] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.127382 29640 ts_tablet_manager.cc:1929] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.127610 29640 log.cc:1199] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/c323b7bcb671464ea6ff584a633f10c0
I20260320 15:27:40.128043 29640 ts_tablet_manager.cc:1950] T c323b7bcb671464ea6ff584a633f10c0 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:40.128217 28734 ts_tablet_manager.cc:1403] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.001s
I20260320 15:27:40.131958 29678 ts_tablet_manager.cc:1929] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.132521 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 112) took 6231 ms. Trace:
I20260320 15:27:40.133625 28734 raft_consensus.cc:359] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.134013 28734 raft_consensus.cc:385] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.152489 28383 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:40.158749 28383 leader_election.cc:343] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.158885 28383 leader_election.cc:304] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:40.159333 30168 raft_consensus.cc:2749] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:40.159765 29842 raft_consensus.cc:2749] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.160310 28734 raft_consensus.cc:740] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.160636 28734 consensus_queue.cc:260] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.161384 28734 ts_tablet_manager.cc:1434] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260320 15:27:40.161728 28734 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.163568 28734 tablet_bootstrap.cc:654] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.165450 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:40.165769 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:40.165992 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:40.166229 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:40.166282 29643 ts_tablet_manager.cc:1929] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:40.166512 29643 log.cc:1199] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ffe278739bbc4ddab848af9c25241380
I20260320 15:27:40.166682 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:40.167039 29643 ts_tablet_manager.cc:1950] T ffe278739bbc4ddab848af9c25241380 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.167459 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 125) took 5972 ms. Trace:
I20260320 15:27:40.167475 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:40.167815 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:40.167659 29643 rpcz_store.cc:276] 0320 15:27:34.194654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.194714 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:40.069118 (+5874404us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:40.069130 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.109089 (+ 39959us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.166069 (+ 56980us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.167440 (+ 1371us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5874198}
I20260320 15:27:40.168526 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:40.168704 29643 tablet_replica.cc:333] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.168864 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ffe278739bbc4ddab848af9c25241380 (table default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]) successfully deleted
I20260320 15:27:40.168951 29643 raft_consensus.cc:2243] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.169133 29643 raft_consensus.cc:2272] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.169162 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:40.169355 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:40.169762 29643 ts_tablet_manager.cc:1916] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.169791 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:40.132999 30114 consensus_queue.cc:1048] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.135860 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.146522 28736 tablet_bootstrap.cc:492] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.180222 28736 ts_tablet_manager.cc:1403] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.005s
I20260320 15:27:40.180552 29678 log.cc:1199] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:40.181064 29678 ts_tablet_manager.cc:1950] T 26818e620ff649d18b671e4f4c7c7e74 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.181298 28736 raft_consensus.cc:359] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.181511 28736 raft_consensus.cc:385] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.181598 28736 raft_consensus.cc:740] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.181834 28736 consensus_queue.cc:260] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.182209 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 119) took 6035 ms. Trace:
I20260320 15:27:40.182271 28736 ts_tablet_manager.cc:1434] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:40.182385 29678 rpcz_store.cc:276] 0320 15:27:34.146569 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.146924 (+ 355us) service_pool.cc:225] Handling call
0320 15:27:40.085921 (+5938997us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:40.085932 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.091173 (+ 5241us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.131791 (+ 40618us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.182150 (+ 50359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5931422}
I20260320 15:27:40.182705 28736 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.182941 29678 tablet_replica.cc:333] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.183203 29678 raft_consensus.cc:2243] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.183983 29678 raft_consensus.cc:2272] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.184557 28736 tablet_bootstrap.cc:654] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.184774 30114 raft_consensus.cc:493] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.184890 29678 ts_tablet_manager.cc:1916] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.184921 30114 raft_consensus.cc:515] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.185588 30114 leader_election.cc:290] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.186295 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.186559 28444 raft_consensus.cc:2393] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:40.187199 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:40.147579 28733 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.187755 28733 ts_tablet_manager.cc:1403] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.162s user 0.009s sys 0.001s
I20260320 15:27:40.150285 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fa5f6fe801b4452bf294a9ae23bb163" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.152129 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.156715 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80810790e42947a69ce6140feb9f7cbc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:40.188751 28584 raft_consensus.cc:2468] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:40.188683 28733 raft_consensus.cc:359] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.188886 28733 raft_consensus.cc:385] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.188969 28733 raft_consensus.cc:740] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.189183 28733 consensus_queue.cc:260] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.189843 28733 ts_tablet_manager.cc:1434] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:40.190200 28733 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.192786 28733 tablet_bootstrap.cc:654] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.194945 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:40.187644 29640 rpcz_store.cc:276] 0320 15:27:33.901131 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.901193 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:40.062939 (+6161746us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:40.062951 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.094212 (+ 31261us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.127232 (+ 33020us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.132471 (+ 5239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6155963}
I20260320 15:27:40.200412 28734 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.200588 28734 ts_tablet_manager.cc:1403] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.003s
W20260320 15:27:40.201020 28524 leader_election.cc:343] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.201357 28734 raft_consensus.cc:359] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.201541 28734 raft_consensus.cc:385] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.201522 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 115) took 6293 ms. Trace:
I20260320 15:27:40.201589 28734 raft_consensus.cc:740] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.201628 29640 rpcz_store.cc:276] 0320 15:27:33.912740 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.912955 (+ 215us) service_pool.cc:225] Handling call
0320 15:27:40.201351 (+6288396us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:40.201366 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.201474 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6282744}
I20260320 15:27:40.201777 28734 consensus_queue.cc:260] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.201960 29640 tablet_replica.cc:333] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:40.202247 28734 ts_tablet_manager.cc:1434] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:40.202280 29640 raft_consensus.cc:2243] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.202514 29640 raft_consensus.cc:2272] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.202546 28734 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.203239 29640 ts_tablet_manager.cc:1916] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.206395 29643 ts_tablet_manager.cc:1929] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.206324 28239 leader_election.cc:304] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:40.206521 29643 log.cc:1199] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:40.207664 28734 tablet_bootstrap.cc:654] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.201325 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 26818e620ff649d18b671e4f4c7c7e74 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
I20260320 15:27:40.201952 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet c323b7bcb671464ea6ff584a633f10c0 (table default.Table_b20f0890d0a046b8b1ec948a07e22f15 [id=ee674a784e8b440aa17e4452dea07dab]) successfully deleted
W20260320 15:27:40.209254 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cea3bf14653f4555acfed1577a79ee4f already in progress: creating tablet
I20260320 15:27:40.209873 30114 raft_consensus.cc:3060] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.212582 30114 raft_consensus.cc:2749] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.209254 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:40.213815 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:40.213994 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:40.214574 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:40.214742 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:40.219648 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bd3032950254e6f9db8da35dbb1acc5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.214984 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:40.220925 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:40.222302 29678 ts_tablet_manager.cc:1929] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.222466 29678 log.cc:1199] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:40.222939 29678 ts_tablet_manager.cc:1950] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.224007 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 127) took 6064 ms. Trace:
I20260320 15:27:40.224172 29678 rpcz_store.cc:276] 0320 15:27:34.159454 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.159508 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:40.182865 (+6023357us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:40.182874 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.211323 (+ 28449us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.222109 (+ 10786us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.223946 (+ 1837us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6022995}
I20260320 15:27:40.224808 29678 tablet_replica.cc:333] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.225107 29678 raft_consensus.cc:2243] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.225334 29678 raft_consensus.cc:2272] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.226590 29640 ts_tablet_manager.cc:1929] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.226732 29640 log.cc:1199] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:40.227125 29640 ts_tablet_manager.cc:1950] T 591ccc1d349e4fd599d5e343f91defe2 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:40.228371 28733 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.228525 28733 ts_tablet_manager.cc:1403] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.007s
I20260320 15:27:40.221629 28905 catalog_manager.cc:5671] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:40.230759 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:40.231426 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:40.232036 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:40.232733 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:40.233024 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:40.233340 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:40.233595 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:40.233845 29678 ts_tablet_manager.cc:1916] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.233899 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:40.234090 28905 catalog_manager.cc:5671] T 685e581b3f084d4a9aa97fba0a7767c7 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:40.234647 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:40.234858 28905 catalog_manager.cc:5671] T f05205970cd249b68d138b904ba61c9e P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:40.238353 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:40.251624 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:40.251714 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]) successfully deleted
I20260320 15:27:40.252082 28905 catalog_manager.cc:5671] T 54a372bb37de43938f0420bdc9c7249c P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:40.252892 28733 raft_consensus.cc:359] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.253111 28733 raft_consensus.cc:385] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.253188 28733 raft_consensus.cc:740] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.253403 28733 consensus_queue.cc:260] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.253969 28733 ts_tablet_manager.cc:1434] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.025s user 0.000s sys 0.001s
I20260320 15:27:40.254287 28733 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.255915 29643 ts_tablet_manager.cc:1950] T 1c2d7e4d26994a17bf2d2615d6552412 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.256525 28733 tablet_bootstrap.cc:654] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.258229 29678 ts_tablet_manager.cc:1929] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:40.258385 29678 log.cc:1199] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ffe278739bbc4ddab848af9c25241380
I20260320 15:27:40.258719 29678 ts_tablet_manager.cc:1950] T ffe278739bbc4ddab848af9c25241380 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.258725 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 116) took 6317 ms. Trace:
I20260320 15:27:40.260545 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 133) took 6098 ms. Trace:
I20260320 15:27:40.260771 29678 rpcz_store.cc:276] 0320 15:27:34.161106 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.161154 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:40.224709 (+6063555us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:40.224725 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.254355 (+ 29630us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.258065 (+ 3710us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.259958 (+ 1893us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6063323}
I20260320 15:27:40.261024 29640 rpcz_store.cc:276] 0320 15:27:33.941130 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.941184 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:40.201895 (+6260711us) ts_tablet_manager.cc:1151] Deleting tablet 591ccc1d349e4fd599d5e343f91defe2
0320 15:27:40.201905 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.223382 (+ 21477us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.226463 (+ 3081us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.258670 (+ 32207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6257292}
I20260320 15:27:40.261540 29678 tablet_replica.cc:333] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.261814 29678 raft_consensus.cc:2243] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:40.261989 29640 tablet_replica.cc:333] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:40.262279 29678 raft_consensus.cc:2272] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.262471 29640 raft_consensus.cc:2243] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.262944 29678 ts_tablet_manager.cc:1916] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.262974 29640 raft_consensus.cc:2272] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.263633 29640 ts_tablet_manager.cc:1916] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.264055 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 123) took 6069 ms. Trace:
I20260320 15:27:40.264168 29643 rpcz_store.cc:276] 0320 15:27:34.194433 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.194797 (+ 364us) service_pool.cc:225] Handling call
0320 15:27:40.168580 (+5973783us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:40.168592 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.172910 (+ 4318us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.206281 (+ 33371us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.264022 (+ 57741us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5973599}
I20260320 15:27:40.264523 29643 tablet_replica.cc:333] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.264712 29643 raft_consensus.cc:2243] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.264863 29643 raft_consensus.cc:2272] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.265697 29643 ts_tablet_manager.cc:1916] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.269414 29678 ts_tablet_manager.cc:1929] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.269965 29678 log.cc:1199] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/3e7b964395f24e45bc2715be3fac6dbe
I20260320 15:27:40.270507 29678 ts_tablet_manager.cc:1950] T 3e7b964395f24e45bc2715be3fac6dbe P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.270294 28239 leader_election.cc:304] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:40.271678 30114 raft_consensus.cc:3060] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.273595 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 117) took 6127 ms. Trace:
I20260320 15:27:40.273764 29678 rpcz_store.cc:276] 0320 15:27:34.146307 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.146987 (+ 680us) service_pool.cc:225] Handling call
0320 15:27:40.261464 (+6114477us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:40.261475 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.266378 (+ 4903us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.269294 (+ 2916us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.273538 (+ 4244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6098171}
I20260320 15:27:40.274340 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 124) took 6124 ms. Trace:
I20260320 15:27:40.274501 29678 rpcz_store.cc:276] 0320 15:27:34.149892 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.149952 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:40.274140 (+6124188us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:40.274150 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.274301 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6110204}
I20260320 15:27:40.274844 29678 tablet_replica.cc:333] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.275067 30114 raft_consensus.cc:2749] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.275259 29678 raft_consensus.cc:2243] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.275498 29678 raft_consensus.cc:2272] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.275966 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 591ccc1d349e4fd599d5e343f91defe2 (table default.Table_ce377d5188904a50803bb93b251ac6aa [id=ab1248ac3aab491b8dd0fc75d85cef24]) successfully deleted
I20260320 15:27:40.276311 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1c2d7e4d26994a17bf2d2615d6552412 (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]) successfully deleted
I20260320 15:27:40.276353 28905 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
W20260320 15:27:40.276914 28892 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 514e7b02a28142c2b371644089044a9d consensus state has no leader
I20260320 15:27:40.277137 28892 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 87) took 4474 ms. Trace:
I20260320 15:27:40.277273 29678 ts_tablet_manager.cc:1916] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.277240 28892 rpcz_store.cc:276] 0320 15:27:35.802772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.840710 (+4037938us) service_pool.cc:225] Handling call
0320 15:27:39.841152 (+ 442us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:39.841441 (+ 289us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:39.841630 (+ 189us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:39.841722 (+ 92us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:39.855286 (+ 13564us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:40.277080 (+421794us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.845356 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.845468 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.845474 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:39.845476 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.845569 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.845583 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.845591 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.845659 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:39.845661 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:39.845696 (+ 35us) write_op.cc:270] Start()
0320 15:27:39.845732 (+ 36us) write_op.cc:276] Timestamp: P: 1774020459845657 usec, L: 0
0320 15:27:39.845737 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.845912 (+ 175us) log.cc:844] Serialized 405 byte log entry
0320 15:27:39.854619 (+ 8707us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.854762 (+ 143us) write_op.cc:301] APPLY: starting
0320 15:27:39.854778 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.854824 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.854854 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.854900 (+ 46us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.854938 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.854946 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.855025 (+ 79us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.855118 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.855139 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:39.855147 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.855246 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.855266 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":502,"num_ops":1,"prepare.queue_time_us":3122,"prepare.run_cpu_time_us":641,"prepare.run_wall_time_us":642,"replication_time_us":8843}]]}
I20260320 15:27:40.278982 28736 tablet_bootstrap.cc:492] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.279134 28736 ts_tablet_manager.cc:1403] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.096s user 0.009s sys 0.010s
I20260320 15:27:40.280131 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ffe278739bbc4ddab848af9c25241380 (table default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]) successfully deleted
I20260320 15:27:40.280762 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 3e7b964395f24e45bc2715be3fac6dbe (table default.Table_c783052b0eb045ea95d77f3722e1ca8b [id=eba1ff6618e0486898840ab0fd2f0882]) successfully deleted
W20260320 15:27:40.281266 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6292904782fd4b619483135e58b66684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6292904782fd4b619483135e58b66684 already in progress: creating tablet
I20260320 15:27:40.282938 28736 raft_consensus.cc:359] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.283159 28736 raft_consensus.cc:385] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.283263 28736 raft_consensus.cc:740] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.283470 28736 consensus_queue.cc:260] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.284276 28736 ts_tablet_manager.cc:1434] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:40.284583 28736 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.291462 30153 raft_consensus.cc:493] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.291684 30153 raft_consensus.cc:515] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.291736 28901 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_313ec95c683c4f99adb04f3968de7963 [id=56aecaca2e3a4c868f419fa30e404f7d] alter complete (version 4)
I20260320 15:27:40.292526 30153 leader_election.cc:290] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.290494 28736 tablet_bootstrap.cc:654] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.299305 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:40.300819 28524 leader_election.cc:343] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.302191 29678 ts_tablet_manager.cc:1929] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.302440 29678 log.cc:1199] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/533105ad6e054fe3a4db32855bc688bd
I20260320 15:27:40.302958 29678 ts_tablet_manager.cc:1950] T 533105ad6e054fe3a4db32855bc688bd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:40.303421 28239 consensus_peers.cc:597] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:40.305172 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 135) took 6141 ms. Trace:
I20260320 15:27:40.306006 29678 rpcz_store.cc:276] 0320 15:27:34.163689 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.163741 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:40.274770 (+6111029us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:40.274779 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.291218 (+ 16439us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.298065 (+ 6847us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.305120 (+ 7055us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6110785}
I20260320 15:27:40.307133 29678 tablet_replica.cc:333] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.308167 29678 raft_consensus.cc:2243] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.308357 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 533105ad6e054fe3a4db32855bc688bd (table default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]) successfully deleted
I20260320 15:27:40.308682 29678 raft_consensus.cc:2272] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.310087 29678 ts_tablet_manager.cc:1916] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.309490 28736 tablet_bootstrap.cc:492] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.311698 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.310678 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.311339 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.311990 28736 ts_tablet_manager.cc:1403] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.000s
I20260320 15:27:40.312171 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.312615 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.313344 28736 raft_consensus.cc:359] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.313686 28736 raft_consensus.cc:385] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.313822 28736 raft_consensus.cc:740] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.314034 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.314297 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.314188 28736 consensus_queue.cc:260] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.317983 28736 ts_tablet_manager.cc:1434] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:40.320412 28736 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.320487 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet bcec6175c1ca49f5b682a29d5c1e235a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.320731 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet f70530cd63a0420fb51527759c951e31 with delete_type TABLET_DATA_DELETED (replaced by tablet c3bc4dd47817490d88725127b456cc70) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.320794 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet c323b7bcb671464ea6ff584a633f10c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.321050 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet bb7a910bd2904a40ae63608377206c7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.328384 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.328938 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.329360 28736 tablet_bootstrap.cc:654] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.331751 28890 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:40.333253 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.334717 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.335687 28903 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_36d13972559c46e7861b4c3a0fbbf8b3 [id=d272c148809a44bea0d673f063e2d677] alter complete (version 1)
I20260320 15:27:40.341822 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet b1bc86d3d1ff4b3aa8463eb21f619917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.342577 28890 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 86) took 4529 ms. Trace:
I20260320 15:27:40.342705 28890 rpcz_store.cc:276] 0320 15:27:35.802619 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.786622 (+3984003us) service_pool.cc:225] Handling call
0320 15:27:39.791782 (+ 5160us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:39.792026 (+ 244us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:39.792127 (+ 101us) catalog_manager.cc:2824] Locking tablets
0320 15:27:39.839785 (+ 47658us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:39.860711 (+ 20926us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.860719 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:39.860812 (+ 93us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:39.860955 (+ 143us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:40.330732 (+469777us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:40.332299 (+ 1567us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.840691 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.840823 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.840829 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:39.840831 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.840956 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.840971 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.840976 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:39.841119 (+ 143us) tablet.cc:700] Row locks acquired
0320 15:27:39.841124 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:39.841183 (+ 59us) write_op.cc:270] Start()
0320 15:27:39.841228 (+ 45us) write_op.cc:276] Timestamp: P: 1774020459841126 usec, L: 0
0320 15:27:39.841235 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.841443 (+ 208us) log.cc:844] Serialized 765 byte log entry
0320 15:27:39.847209 (+ 5766us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.847339 (+ 130us) write_op.cc:301] APPLY: starting
0320 15:27:39.847353 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.847401 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.847429 (+ 28us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.847502 (+ 73us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.847578 (+ 76us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:39.847586 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.847658 (+ 72us) log.cc:844] Serialized 29 byte log entry
0320 15:27:39.847721 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.847738 (+ 17us) write_op.cc:454] Released schema lock
0320 15:27:39.847743 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":462,"apply.run_wall_time_us":462,"num_ops":2,"prepare.queue_time_us":68,"prepare.run_cpu_time_us":862,"prepare.run_wall_time_us":863,"raft.queue_time_us":652,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":385,"replication_time_us":5940}]]}
W20260320 15:27:40.344434 29807 log.cc:927] Time spent T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20260320 15:27:40.345264 28733 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.345531 28733 ts_tablet_manager.cc:1403] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
W20260320 15:27:40.349902 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 117) took 3775 ms (client timeout 1000 ms). Trace:
I20260320 15:27:40.350559 28733 raft_consensus.cc:359] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.350905 28733 raft_consensus.cc:385] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.351043 28733 raft_consensus.cc:740] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.351389 28733 consensus_queue.cc:260] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.365000 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 591ccc1d349e4fd599d5e343f91defe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:21 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.365835 30114 consensus_queue.cc:1048] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.367460 28733 ts_tablet_manager.cc:1434] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260320 15:27:40.369208 28733 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:40.371752 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 103) took 2474 ms (client timeout 1000 ms). Trace:
I20260320 15:27:40.373816 30168 raft_consensus.cc:493] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.373971 30168 raft_consensus.cc:515] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.374629 30168 leader_election.cc:290] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.381256 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:40.381862 28382 leader_election.cc:343] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.382107 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4b53fb135741efad5d3e59a6d4eb3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.382715 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.383041 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet a4bfb87d5d5741adb0ad2f2a88f4867f with delete_type TABLET_DATA_DELETED (replaced by tablet 514e7b02a28142c2b371644089044a9d) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.383222 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet f5f92561764b4cb9b15bb9a310a350f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.383412 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet df5ffaedeaf94def93d93a538c42e3bd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a072859f39c4c28ae65b0e9afa79d66) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.383651 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c2d7e4d26994a17bf2d2615d6552412 with delete_type TABLET_DATA_DELETED (replaced by tablet 4bd1799035a3431092fbdfad3c42d22b) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.383899 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 89425da569794a8fa972817023bf853d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.384059 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.384250 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.384428 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.384572 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.385583 29618 consensus_queue.cc:1048] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.386593 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.386650 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.386830 28444 raft_consensus.cc:2468] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:40.389758 30153 raft_consensus.cc:493] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.390036 30153 raft_consensus.cc:515] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.390764 30153 leader_election.cc:290] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.391228 28733 tablet_bootstrap.cc:654] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.391629 28524 leader_election.cc:304] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:40.392122 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.392215 29939 raft_consensus.cc:2804] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.392300 29939 raft_consensus.cc:493] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.392383 29939 raft_consensus.cc:3060] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:40.392745 28383 leader_election.cc:343] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.392868 28383 leader_election.cc:304] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:40.393332 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.393517 28445 raft_consensus.cc:2468] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:40.393503 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:40.394341 28524 leader_election.cc:343] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.394737 28524 leader_election.cc:304] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:40.395200 30153 raft_consensus.cc:2804] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.395298 30153 raft_consensus.cc:493] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.395375 30153 raft_consensus.cc:3060] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.395648 30168 raft_consensus.cc:2749] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.395408 29939 raft_consensus.cc:515] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.398556 29939 leader_election.cc:290] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.399590 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.399811 28295 raft_consensus.cc:3060] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.399920 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f70142c3a449439a0a475bddafdd40" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.400127 28445 raft_consensus.cc:3060] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.402720 28295 raft_consensus.cc:2468] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.403115 28445 raft_consensus.cc:2468] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.403429 28524 leader_election.cc:304] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
W20260320 15:27:40.366806 28906 rpcz_store.cc:269] 0320 15:27:36.554983 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.974849 (+419866us) service_pool.cc:225] Handling call
0320 15:27:40.330364 (+3355515us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:38.579600 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.579750 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.579758 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:38.579760 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:38.579880 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.579901 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.579907 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:38.579975 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:38.579977 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:38.580020 (+ 43us) write_op.cc:270] Start()
0320 15:27:38.580058 (+ 38us) write_op.cc:276] Timestamp: P: 1774020458579959 usec, L: 0
0320 15:27:38.580064 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.580322 (+ 258us) log.cc:844] Serialized 527 byte log entry
0320 15:27:38.585533 (+ 5211us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.585686 (+ 153us) write_op.cc:301] APPLY: starting
0320 15:27:38.585711 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.585769 (+ 58us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.585831 (+ 62us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.585909 (+ 78us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.585977 (+ 68us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:38.585989 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:38.586109 (+ 120us) log.cc:844] Serialized 23 byte log entry
0320 15:27:38.586202 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.586234 (+ 32us) write_op.cc:454] Released schema lock
0320 15:27:38.586243 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:38.586339 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.586345 (+ 6us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:38.791769 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.791937 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.791946 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:38.791948 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:38.792074 (+ 126us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.792091 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.792096 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:38.792163 (+ 67us) tablet.cc:700] Row locks acquired
0320 15:27:38.792166 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:38.792213 (+ 47us) write_op.cc:270] Start()
0320 15:27:38.792259 (+ 46us) write_op.cc:276] Timestamp: P: 1774020458792161 usec, L: 0
0320 15:27:38.792264 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.792539 (+ 275us) log.cc:844] Serialized 356 byte log entry
0320 15:27:38.797689 (+ 5150us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.797825 (+ 136us) write_op.cc:301] APPLY: starting
0320 15:27:38.797848 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.797895 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.797936 (+ 41us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.797997 (+ 61us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.798036 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:38.798044 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:38.798134 (+ 90us) log.cc:844] Serialized 23 byte log entry
0320 15:27:38.798205 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.798228 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:38.798236 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:38.798334 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.798338 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":302465,"reported_tablets":216,"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":638,"apply.run_wall_time_us":637,"num_ops":1,"prepare.queue_time_us":206,"prepare.run_cpu_time_us":844,"prepare.run_wall_time_us":844,"raft.queue_time_us":448,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":267,"replication_time_us":5410}],["op",{"apply.queue_time_us":76,"apply.run_cpu_time_us":499,"apply.run_wall_time_us":499,"num_ops":1,"prepare.queue_time_us":6865,"prepare.run_cpu_time_us":915,"prepare.run_wall_time_us":917,"raft.queue_time_us":489,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":263,"replication_time_us":5388}]]}
I20260320 15:27:40.410108 28736 tablet_bootstrap.cc:492] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.410665 28736 ts_tablet_manager.cc:1403] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.090s user 0.004s sys 0.001s
I20260320 15:27:40.410974 29939 raft_consensus.cc:2804] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.411477 29939 raft_consensus.cc:697] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:40.411881 29939 consensus_queue.cc:237] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.413631 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.414016 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 66a181681cd84b7fb09c3742f4f2222f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.415727 29678 ts_tablet_manager.cc:1929] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.415871 29678 log.cc:1199] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b6201b74fdb0428e98f2eebb0b24d01a
I20260320 15:27:40.416014 30168 raft_consensus.cc:493] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.416239 29678 ts_tablet_manager.cc:1950] T b6201b74fdb0428e98f2eebb0b24d01a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.416167 30168 raft_consensus.cc:515] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.417135 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 123) took 6267 ms. Trace:
I20260320 15:27:40.417241 29678 rpcz_store.cc:276] 0320 15:27:34.149476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.149602 (+ 126us) service_pool.cc:225] Handling call
0320 15:27:40.307046 (+6157444us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:40.307059 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.367354 (+ 60295us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.415567 (+ 48213us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.417089 (+ 1522us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6142858}
I20260320 15:27:40.417639 29678 tablet_replica.cc:333] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.417687 30168 leader_election.cc:290] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:40.417840 29678 raft_consensus.cc:2243] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:40.418141 29678 raft_consensus.cc:2272] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.418299 30153 raft_consensus.cc:515] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.418606 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.418774 29678 ts_tablet_manager.cc:1916] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.418951 30153 leader_election.cc:290] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:40.419291 28382 leader_election.cc:343] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.420565 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.420994 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.423133 29643 ts_tablet_manager.cc:1929] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.423264 29643 log.cc:1199] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/533105ad6e054fe3a4db32855bc688bd
I20260320 15:27:40.423717 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60d0b36f462945cbbca7e389edf494d3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.423938 28442 raft_consensus.cc:3060] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:40.425892 28383 leader_election.cc:343] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.426030 28383 leader_election.cc:304] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:40.426527 30168 raft_consensus.cc:2749] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.426892 28442 raft_consensus.cc:2468] T 60d0b36f462945cbbca7e389edf494d3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.413466 28734 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.427685 28734 ts_tablet_manager.cc:1403] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.225s user 0.006s sys 0.030s
W20260320 15:27:40.427594 28524 leader_election.cc:343] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.428567 28734 raft_consensus.cc:359] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.428856 28734 raft_consensus.cc:385] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.428946 28734 raft_consensus.cc:740] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.429231 28734 consensus_queue.cc:260] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.429839 28734 ts_tablet_manager.cc:1434] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:40.430233 28734 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.430089 28524 leader_election.cc:304] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:40.432206 28734 tablet_bootstrap.cc:654] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.432368 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d021f6d697df4823968c7f3b2d5d779c" } new_table_name: "default.Table_07e18d8b90c34136804ef8e64f659f82" modify_external_catalogs: true
I20260320 15:27:40.432502 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b6201b74fdb0428e98f2eebb0b24d01a (table default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]) successfully deleted
W20260320 15:27:40.433243 28900 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 116) took 5053 ms (client timeout 1000 ms). Trace:
I20260320 15:27:40.433532 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
W20260320 15:27:40.414530 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 93) took 5093 ms (client timeout 1000 ms). Trace:
W20260320 15:27:40.434275 28900 rpcz_store.cc:269] 0320 15:27:35.360384 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.594552 (+234168us) service_pool.cc:225] Handling call
0320 15:27:40.413412 (+4818860us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66300,"reported_tablets":210}
I20260320 15:27:40.423713 29643 ts_tablet_manager.cc:1950] T 533105ad6e054fe3a4db32855bc688bd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.435909 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 127) took 6241 ms. Trace:
I20260320 15:27:40.435971 28900 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_37e1fe279d384188986393c967dcb9de" } modify_external_catalogs: true
I20260320 15:27:40.436223 29643 rpcz_store.cc:276] 0320 15:27:34.194951 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195053 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:40.264469 (+6069416us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:40.264483 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.269130 (+ 4647us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.423006 (+153876us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.435886 (+ 12880us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6069211}
I20260320 15:27:40.436980 29643 tablet_replica.cc:333] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.437263 29643 raft_consensus.cc:2243] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.437510 29643 raft_consensus.cc:2272] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.438161 29643 ts_tablet_manager.cc:1916] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.442162 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 533105ad6e054fe3a4db32855bc688bd (table default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]) successfully deleted
I20260320 15:27:40.443614 30153 raft_consensus.cc:2804] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.443785 30153 raft_consensus.cc:697] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
W20260320 15:27:40.380172 28901 rpcz_store.cc:269] 0320 15:27:37.868194 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.535346 (+667152us) service_pool.cc:225] Handling call
0320 15:27:40.342459 (+1807113us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:38.975891 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:38.976053 (+ 162us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:38.976062 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:38.976064 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:38.976244 (+ 180us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:38.976268 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:38.976277 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:38.976425 (+ 148us) tablet.cc:700] Row locks acquired
0320 15:27:38.976427 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:38.976474 (+ 47us) write_op.cc:270] Start()
0320 15:27:38.976522 (+ 48us) write_op.cc:276] Timestamp: P: 1774020458976405 usec, L: 0
0320 15:27:38.976525 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:38.976807 (+ 282us) log.cc:844] Serialized 901 byte log entry
0320 15:27:38.982757 (+ 5950us) op_driver.cc:464] REPLICATION: finished
0320 15:27:38.982880 (+ 123us) write_op.cc:301] APPLY: starting
0320 15:27:38.982900 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:38.982953 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:38.982985 (+ 32us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:38.983060 (+ 75us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:38.983096 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:38.983104 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:38.983192 (+ 88us) log.cc:844] Serialized 29 byte log entry
0320 15:27:38.983263 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.983288 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:38.983296 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:38.983383 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:38.983387 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:39.283685 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.283818 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.283827 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:39.283829 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:39.283942 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.283964 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.283969 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.284039 (+ 70us) tablet.cc:700] Row locks acquired
0320 15:27:39.284041 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:39.284082 (+ 41us) write_op.cc:270] Start()
0320 15:27:39.284123 (+ 41us) write_op.cc:276] Timestamp: P: 1774020459284046 usec, L: 0
0320 15:27:39.284127 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.284349 (+ 222us) log.cc:844] Serialized 356 byte log entry
0320 15:27:39.290272 (+ 5923us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.290422 (+ 150us) write_op.cc:301] APPLY: starting
0320 15:27:39.290442 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.290490 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.290648 (+ 158us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.290734 (+ 86us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.290775 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.290783 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.290882 (+ 99us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.290963 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.291005 (+ 42us) write_op.cc:454] Released schema lock
0320 15:27:39.291013 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.291409 (+ 396us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.291428 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49421,"reported_tablets":207,"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":492,"num_ops":2,"prepare.queue_time_us":1005,"prepare.run_cpu_time_us":1074,"prepare.run_wall_time_us":1071,"raft.queue_time_us":981,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":319,"replication_time_us":6193}],["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":963,"apply.run_wall_time_us":963,"num_ops":1,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":788,"prepare.run_wall_time_us":787,"raft.queue_time_us":671,"raft.run_cpu_time_us":435,"raft.run_wall_time_us":437,"replication_time_us":6098}]]}
I20260320 15:27:40.444038 30153 consensus_queue.cc:237] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.445786 28901 master_service.cc:954] ReplaceTablet: received request to replace tablet 3ed29146bff74e86a366d3c6329176f4 from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:40.453954 28902 rpcz_store.cc:269] 0320 15:27:35.312707 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.594541 (+281834us) service_pool.cc:225] Handling call
0320 15:27:40.405807 (+4811266us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:36.645652 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:36.646935 (+ 1283us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:36.646945 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:36.646948 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:36.649446 (+ 2498us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:36.649557 (+ 111us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:36.649564 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:36.649944 (+ 380us) tablet.cc:700] Row locks acquired
0320 15:27:36.649951 (+ 7us) write_op.cc:260] PREPARE: finished
0320 15:27:36.650095 (+ 144us) write_op.cc:270] Start()
0320 15:27:36.650232 (+ 137us) write_op.cc:276] Timestamp: P: 1774020456650046 usec, L: 0
0320 15:27:36.650345 (+ 113us) op_driver.cc:348] REPLICATION: starting
0320 15:27:36.651284 (+ 939us) log.cc:844] Serialized 527 byte log entry
0320 15:27:36.738887 (+ 87603us) op_driver.cc:464] REPLICATION: finished
0320 15:27:36.764723 (+ 25836us) write_op.cc:301] APPLY: starting
0320 15:27:36.764747 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:36.765011 (+ 264us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:36.765151 (+ 140us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:36.765638 (+ 487us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:36.765789 (+ 151us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:36.765906 (+ 117us) write_op.cc:312] APPLY: finished
0320 15:27:36.766495 (+ 589us) log.cc:844] Serialized 23 byte log entry
0320 15:27:36.767046 (+ 551us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:36.767073 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:36.767185 (+ 112us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:36.767590 (+ 405us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:36.767597 (+ 7us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:37.163301 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:37.163396 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:37.163402 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:37.163403 (+ 1us) tablet.cc:661] Decoding operations
0320 15:27:37.163493 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:37.163506 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:37.163511 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:37.163605 (+ 94us) tablet.cc:700] Row locks acquired
0320 15:27:37.163607 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:37.163638 (+ 31us) write_op.cc:270] Start()
0320 15:27:37.163666 (+ 28us) write_op.cc:276] Timestamp: P: 1774020457163598 usec, L: 0
0320 15:27:37.163669 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:37.163815 (+ 146us) log.cc:844] Serialized 356 byte log entry
0320 15:27:37.175945 (+ 12130us) op_driver.cc:464] REPLICATION: finished
0320 15:27:37.176513 (+ 568us) write_op.cc:301] APPLY: starting
0320 15:27:37.176534 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:37.176583 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:37.176613 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:37.176677 (+ 64us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:37.176715 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:37.176726 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:37.221719 (+ 44993us) log.cc:844] Serialized 23 byte log entry
0320 15:27:37.225400 (+ 3681us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.225422 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:37.225428 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:37.225540 (+ 112us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:37.225546 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":70282,"reported_tablets":215,"child_traces":[["op",{"apply.queue_time_us":25421,"apply.run_cpu_time_us":1021,"apply.run_wall_time_us":2877,"num_ops":1,"prepare.queue_time_us":55161,"prepare.run_cpu_time_us":1387,"prepare.run_wall_time_us":29612,"raft.queue_time_us":200,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":7310,"replication_time_us":88265,"thread_start_us":81606,"threads_started":2}],["op",{"apply.queue_time_us":490,"apply.run_cpu_time_us":609,"apply.run_wall_time_us":49015,"num_ops":1,"prepare.queue_time_us":466,"prepare.run_cpu_time_us":565,"prepare.run_wall_time_us":564,"replication_time_us":12241}]]}
I20260320 15:27:40.456183 28900 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_37e1fe279d384188986393c967dcb9de" } modify_external_catalogs: true
I20260320 15:27:40.459028 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ab9893da469449bba556d11bd347d89b (DEFAULT_TABLE table=default.Table_1b61db70cc644eaeb675dfb95960f295 [id=910ef5cd98e1495f829465985895c17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.459182 28413 tablet_service.cc:1511] Processing CreateTablet for tablet ab9893da469449bba556d11bd347d89b (DEFAULT_TABLE table=default.Table_1b61db70cc644eaeb675dfb95960f295 [id=910ef5cd98e1495f829465985895c17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.459548 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab9893da469449bba556d11bd347d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.459662 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab9893da469449bba556d11bd347d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.463965 28280 tablet_service.cc:1511] Processing CreateTablet for tablet ab9893da469449bba556d11bd347d89b (DEFAULT_TABLE table=default.Table_1b61db70cc644eaeb675dfb95960f295 [id=910ef5cd98e1495f829465985895c17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.464471 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab9893da469449bba556d11bd347d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.467877 28901 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 90) took 4108 ms. Trace:
I20260320 15:27:40.468099 28901 rpcz_store.cc:276] 0320 15:27:36.359510 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.444141 (+4084631us) service_pool.cc:225] Handling call
0320 15:27:40.467823 (+ 23682us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:40.468475 28901 master_service.cc:954] ReplaceTablet: received request to replace tablet b03e7555b9a646b599acb0c7dff90c9a from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:40.468993 28901 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 92) took 3863 ms. Trace:
I20260320 15:27:40.469151 28901 rpcz_store.cc:276] 0320 15:27:36.605212 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.468392 (+3863180us) service_pool.cc:225] Handling call
0320 15:27:40.468929 (+ 537us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260320 15:27:40.469857 28524 consensus_peers.cc:597] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe278739bbc4ddab848af9c25241380. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:40.470916 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b (DEFAULT_TABLE table=default.Table_1125a332fd6d45dfbd5dfbd174357989 [id=7d1d5827aa8a4e36bc634fda386ea662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.471347 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b (DEFAULT_TABLE table=default.Table_1125a332fd6d45dfbd5dfbd174357989 [id=7d1d5827aa8a4e36bc634fda386ea662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.471480 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.471494 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b (DEFAULT_TABLE table=default.Table_1125a332fd6d45dfbd5dfbd174357989 [id=7d1d5827aa8a4e36bc634fda386ea662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.471796 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.471899 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c4fa63d52b24d45b4fe0a2e0842bd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.479741 28902 master_service.cc:954] ReplaceTablet: received request to replace tablet 3ed5d220ec4b406aa5bae970f785de63 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:40.479931 30153 raft_consensus.cc:493] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.480072 30153 raft_consensus.cc:515] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.480739 30153 leader_election.cc:290] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.481748 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.482028 28445 raft_consensus.cc:2468] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:40.482810 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc1e90717f49c2b784391c8de04c37 (DEFAULT_TABLE table=default.Table_39acdde6b06e4e4e9a09138804663dba [id=284293f3ec024c29b48fcad65f40a925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.483400 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc1e90717f49c2b784391c8de04c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.484145 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:40.484841 28524 leader_election.cc:343] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.488637 28524 leader_election.cc:304] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:40.490054 30153 raft_consensus.cc:2804] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.490193 30153 raft_consensus.cc:493] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.490269 30153 raft_consensus.cc:3060] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.500809 30153 raft_consensus.cc:515] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.503176 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.503397 28445 raft_consensus.cc:3060] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.503432 28733 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.503607 28733 ts_tablet_manager.cc:1403] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.135s user 0.007s sys 0.000s
I20260320 15:27:40.504442 30153 leader_election.cc:290] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.504539 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.504842 28733 raft_consensus.cc:359] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.505009 28733 raft_consensus.cc:385] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.505056 28733 raft_consensus.cc:740] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.505170 28736 raft_consensus.cc:359] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.505254 28733 consensus_queue.cc:260] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.505359 28736 raft_consensus.cc:385] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.505448 28736 raft_consensus.cc:740] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.505693 28736 consensus_queue.cc:260] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.505859 28733 ts_tablet_manager.cc:1434] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:40.506145 28733 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.506232 28736 ts_tablet_manager.cc:1434] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.095s user 0.002s sys 0.000s
I20260320 15:27:40.507941 28733 tablet_bootstrap.cc:654] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.507918 28901 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8f15c883f7c74571a7717c5dcdfb4f6d" } new_table_name: "default.Table_13967321267549daa281ef67538d1894" modify_external_catalogs: true
W20260320 15:27:40.509042 28524 leader_election.cc:343] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.511065 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc1e90717f49c2b784391c8de04c37 (DEFAULT_TABLE table=default.Table_39acdde6b06e4e4e9a09138804663dba [id=284293f3ec024c29b48fcad65f40a925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.511493 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc1e90717f49c2b784391c8de04c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.511572 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc1e90717f49c2b784391c8de04c37 (DEFAULT_TABLE table=default.Table_39acdde6b06e4e4e9a09138804663dba [id=284293f3ec024c29b48fcad65f40a925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.512565 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc1e90717f49c2b784391c8de04c37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.511638 28445 raft_consensus.cc:2468] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.513965 28524 leader_election.cc:304] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:40.514258 29599 raft_consensus.cc:2804] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.514382 29599 raft_consensus.cc:697] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:40.514637 29599 consensus_queue.cc:237] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.515884 28280 tablet_service.cc:1511] Processing CreateTablet for tablet c87a1ae2cd1b4f178267594fdf4ce9d8 (DEFAULT_TABLE table=default.Table_cfd32ab97f564c6dbe89916a2960fe1a [id=45f3b6e3bf714f7ea7806f6b04d47127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.518121 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:40.519032 28902 catalog_manager.cc:6741] ReplaceTablet: tablet 3ed5d220ec4b406aa5bae970f785de63 deleted and replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78
I20260320 15:27:40.523239 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 with delete_type TABLET_DATA_DELETED (replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:40.518008 28524 consensus_peers.cc:597] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe278739bbc4ddab848af9c25241380. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:40.520947 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 with delete_type TABLET_DATA_DELETED (replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.524816 28902 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 91) took 4070 ms. Trace:
I20260320 15:27:40.525012 28902 rpcz_store.cc:276] 0320 15:27:36.450398 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.455815 (+4005417us) service_pool.cc:225] Handling call
0320 15:27:40.520688 (+ 64873us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.481470 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.481619 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.481626 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:40.481629 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:40.481761 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.481778 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.481783 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:40.481889 (+ 106us) tablet.cc:700] Row locks acquired
0320 15:27:40.481891 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:40.481935 (+ 44us) write_op.cc:270] Start()
0320 15:27:40.481978 (+ 43us) write_op.cc:276] Timestamp: P: 1774020460481887 usec, L: 0
0320 15:27:40.481981 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.482194 (+ 213us) log.cc:844] Serialized 622 byte log entry
0320 15:27:40.489625 (+ 7431us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.489767 (+ 142us) write_op.cc:301] APPLY: starting
0320 15:27:40.489785 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.489836 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.489888 (+ 52us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.490003 (+ 115us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.490071 (+ 68us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:40.490079 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:40.490177 (+ 98us) log.cc:844] Serialized 29 byte log entry
0320 15:27:40.490245 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.490267 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:40.490275 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.490360 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.490365 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":88,"apply.run_cpu_time_us":582,"apply.run_wall_time_us":582,"num_ops":2,"prepare.queue_time_us":63,"prepare.run_cpu_time_us":850,"prepare.run_wall_time_us":850,"raft.queue_time_us":2599,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":294,"replication_time_us":7608}]]}
I20260320 15:27:40.524672 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 with delete_type TABLET_DATA_DELETED (replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.526808 28559 tablet_service.cc:1511] Processing CreateTablet for tablet c87a1ae2cd1b4f178267594fdf4ce9d8 (DEFAULT_TABLE table=default.Table_cfd32ab97f564c6dbe89916a2960fe1a [id=45f3b6e3bf714f7ea7806f6b04d47127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.528285 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87a1ae2cd1b4f178267594fdf4ce9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.531920 30168 raft_consensus.cc:493] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.532083 30168 raft_consensus.cc:515] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.532809 30168 leader_election.cc:290] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.533762 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.533946 28574 raft_consensus.cc:2468] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:40.534425 28382 leader_election.cc:304] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.534755 30168 raft_consensus.cc:2804] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.534824 30168 raft_consensus.cc:493] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.534878 30168 raft_consensus.cc:3060] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.535600 28902 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_a42ece35a94a4c2b9c3bbfb885208c5b" } modify_external_catalogs: true
I20260320 15:27:40.535811 28902 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_a42ece35a94a4c2b9c3bbfb885208c5b" } modify_external_catalogs: true
I20260320 15:27:40.537806 28421 tablet_service.cc:1511] Processing CreateTablet for tablet c87a1ae2cd1b4f178267594fdf4ce9d8 (DEFAULT_TABLE table=default.Table_cfd32ab97f564c6dbe89916a2960fe1a [id=45f3b6e3bf714f7ea7806f6b04d47127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.537990 28551 tablet_service.cc:1511] Processing CreateTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f (DEFAULT_TABLE table=default.Table_8d69a7f0253d4f47a24fcabe3b0dff20 [id=2e390174c541457e96fcadb41601b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.538285 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87a1ae2cd1b4f178267594fdf4ce9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.538364 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1965a7f25ca4c4d95575a6c93ec376f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.538923 30168 raft_consensus.cc:515] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.540385 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:40.540562 28584 raft_consensus.cc:3060] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.543182 28584 raft_consensus.cc:2468] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
W20260320 15:27:40.544190 29807 log.cc:927] Time spent T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Append to log took a long time: real 0.117s user 0.003s sys 0.000s
I20260320 15:27:40.544526 28382 leader_election.cc:304] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:40.544950 30169 raft_consensus.cc:2804] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.545069 30169 raft_consensus.cc:697] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:40.545328 30169 consensus_queue.cc:237] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.551290 30168 leader_election.cc:290] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.552186 28736 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.552376 28295 raft_consensus.cc:3060] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.554563 28736 tablet_bootstrap.cc:654] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.560103 28268 tablet_service.cc:1511] Processing CreateTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f (DEFAULT_TABLE table=default.Table_8d69a7f0253d4f47a24fcabe3b0dff20 [id=2e390174c541457e96fcadb41601b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.561196 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 158e679a7a214688a3bc1427bbf975b7 (DEFAULT_TABLE table=default.Table_c883a162464b4030a2c38e1eb9b7efe4 [id=233c243527b54a73b249045a6734c475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.561429 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.562036 28421 tablet_service.cc:1511] Processing CreateTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f (DEFAULT_TABLE table=default.Table_8d69a7f0253d4f47a24fcabe3b0dff20 [id=2e390174c541457e96fcadb41601b261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.562388 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158e679a7a214688a3bc1427bbf975b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.562420 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1965a7f25ca4c4d95575a6c93ec376f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.563756 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c87a1ae2cd1b4f178267594fdf4ce9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.564333 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1965a7f25ca4c4d95575a6c93ec376f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.564254 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.568218 28734 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.568368 28734 ts_tablet_manager.cc:1403] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.138s user 0.003s sys 0.000s
I20260320 15:27:40.569147 28734 raft_consensus.cc:359] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.569314 28734 raft_consensus.cc:385] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.569358 28734 raft_consensus.cc:740] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.569559 28734 consensus_queue.cc:260] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.570019 28734 ts_tablet_manager.cc:1434] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:40.570290 28734 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:40.572739 28383 leader_election.cc:343] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:40.575176 28383 leader_election.cc:343] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.579126 29643 ts_tablet_manager.cc:1929] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.579262 29643 log.cc:1199] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7be03035dc3d44f9aa5b43b49281a940
I20260320 15:27:40.581070 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 744a727e9f2e40fea9cef0a05fe22989 (DEFAULT_TABLE table=default.Table_3f812789f8e54c0ab7870ab350295947 [id=588ea40f635a4ed58f1617cbe6663e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.581100 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 62c168a88d3b46f7be2cad36e3385e60 (DEFAULT_TABLE table=default.Table_e0d5290fa4ae42909d4be711b08b4018 [id=86c2fb7b579b45af820e6356728f25a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.584287 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744a727e9f2e40fea9cef0a05fe22989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.587832 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c168a88d3b46f7be2cad36e3385e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.591463 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 674150ada03640c9bc71a29215b49dbb (DEFAULT_TABLE table=default.Table_e708aa24a35c49d88ac8f3e84bcdf4e6 [id=4bfb656bcf814ee784a86990b9091cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.592068 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674150ada03640c9bc71a29215b49dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.597468 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 158e679a7a214688a3bc1427bbf975b7 (DEFAULT_TABLE table=default.Table_c883a162464b4030a2c38e1eb9b7efe4 [id=233c243527b54a73b249045a6734c475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.597945 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158e679a7a214688a3bc1427bbf975b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.600131 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 158e679a7a214688a3bc1427bbf975b7 (DEFAULT_TABLE table=default.Table_c883a162464b4030a2c38e1eb9b7efe4 [id=233c243527b54a73b249045a6734c475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.601114 28422 tablet_service.cc:1511] Processing CreateTablet for tablet 744a727e9f2e40fea9cef0a05fe22989 (DEFAULT_TABLE table=default.Table_3f812789f8e54c0ab7870ab350295947 [id=588ea40f635a4ed58f1617cbe6663e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.601538 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744a727e9f2e40fea9cef0a05fe22989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.605803 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 62c168a88d3b46f7be2cad36e3385e60 (DEFAULT_TABLE table=default.Table_e0d5290fa4ae42909d4be711b08b4018 [id=86c2fb7b579b45af820e6356728f25a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.606146 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 674150ada03640c9bc71a29215b49dbb (DEFAULT_TABLE table=default.Table_e708aa24a35c49d88ac8f3e84bcdf4e6 [id=4bfb656bcf814ee784a86990b9091cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.606271 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c168a88d3b46f7be2cad36e3385e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.606531 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674150ada03640c9bc71a29215b49dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.607928 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 158e679a7a214688a3bc1427bbf975b7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:40.609764 28524 consensus_peers.cc:597] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 583ed053b2a742c7b029f348f4495101. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:40.614421 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 62c168a88d3b46f7be2cad36e3385e60 (DEFAULT_TABLE table=default.Table_e0d5290fa4ae42909d4be711b08b4018 [id=86c2fb7b579b45af820e6356728f25a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.614915 29643 ts_tablet_manager.cc:1950] T 7be03035dc3d44f9aa5b43b49281a940 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.614944 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c168a88d3b46f7be2cad36e3385e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.618604 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 744a727e9f2e40fea9cef0a05fe22989 (DEFAULT_TABLE table=default.Table_3f812789f8e54c0ab7870ab350295947 [id=588ea40f635a4ed58f1617cbe6663e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.619062 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744a727e9f2e40fea9cef0a05fe22989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.615404 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 128) took 6420 ms. Trace:
I20260320 15:27:40.623698 29599 raft_consensus.cc:493] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.623837 29599 raft_consensus.cc:515] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.623659 29643 rpcz_store.cc:276] 0320 15:27:34.195100 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195243 (+ 143us) service_pool.cc:225] Handling call
0320 15:27:40.436925 (+6241682us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:40.436936 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.441509 (+ 4573us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.579015 (+137506us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.615385 (+ 36370us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6241491}
I20260320 15:27:40.624156 29643 tablet_replica.cc:333] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.624361 29643 raft_consensus.cc:2243] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:40.624406 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7be03035dc3d44f9aa5b43b49281a940 (table default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]) successfully deleted
I20260320 15:27:40.625254 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 674150ada03640c9bc71a29215b49dbb (DEFAULT_TABLE table=default.Table_e708aa24a35c49d88ac8f3e84bcdf4e6 [id=4bfb656bcf814ee784a86990b9091cdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.625414 29643 raft_consensus.cc:2272] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.625720 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674150ada03640c9bc71a29215b49dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.626003 29643 ts_tablet_manager.cc:1916] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.628049 29599 leader_election.cc:290] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:40.632288 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:40.634402 28734 tablet_bootstrap.cc:654] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.634752 28736 tablet_bootstrap.cc:492] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.635005 28736 ts_tablet_manager.cc:1403] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.083s user 0.005s sys 0.000s
I20260320 15:27:40.636436 28736 raft_consensus.cc:359] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.636739 28736 raft_consensus.cc:385] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.636826 28733 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.636857 28736 raft_consensus.cc:740] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.636982 28733 ts_tablet_manager.cc:1403] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.131s user 0.004s sys 0.010s
I20260320 15:27:40.637225 28736 consensus_queue.cc:260] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.637809 28736 ts_tablet_manager.cc:1434] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:40.638157 28736 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.638641 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:40.640554 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:40.641952 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:40.642153 28733 raft_consensus.cc:359] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.642396 28733 raft_consensus.cc:385] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.642475 28733 raft_consensus.cc:740] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.642597 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:40.643002 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:40.643170 28733 consensus_queue.cc:260] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.643900 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:40.644783 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:40.652720 28733 ts_tablet_manager.cc:1434] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260320 15:27:40.653118 28733 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.654403 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:40.660732 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
W20260320 15:27:40.663856 28524 leader_election.cc:343] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:40.664026 28524 leader_election.cc:304] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:40.664314 29599 raft_consensus.cc:2749] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:40.664188 28733 tablet_bootstrap.cc:654] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:40.667793 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 104) took 1768 ms (client timeout 1000 ms). Trace:
I20260320 15:27:40.671002 29618 raft_consensus.cc:493] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.671182 29618 raft_consensus.cc:515] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.671490 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.671870 29618 leader_election.cc:290] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:40.674669 28239 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
W20260320 15:27:40.676255 28889 rpcz_store.cc:269] 0320 15:27:38.899404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.997114 (+ 97710us) service_pool.cc:225] Handling call
0320 15:27:40.667730 (+1670616us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13485,"reported_tablets":202}
I20260320 15:27:40.676815 28889 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_80ba3729fee5441b983da0a2323b6ecb" } modify_external_catalogs: true
I20260320 15:27:40.677105 28889 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_80ba3729fee5441b983da0a2323b6ecb" } modify_external_catalogs: true
I20260320 15:27:40.676057 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:40.677922 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:40.678526 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:40.680505 28736 tablet_bootstrap.cc:654] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.680789 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:40.681138 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:40.681793 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:40.682137 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c]). Expected version 1 got 0
I20260320 15:27:40.685570 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:40.689626 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:40.690027 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:40.690972 29678 ts_tablet_manager.cc:1929] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.692875 29678 log.cc:1199] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/aafd1e123cd043e7828140d2443e047f
I20260320 15:27:40.703612 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:40.703641 28240 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:40.703763 28240 leader_election.cc:304] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:40.704456 29618 raft_consensus.cc:2749] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.700716 29678 ts_tablet_manager.cc:1950] T aafd1e123cd043e7828140d2443e047f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.705771 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:40.706174 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:40.706408 29643 ts_tablet_manager.cc:1929] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.706547 29643 log.cc:1199] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:40.710072 28736 tablet_bootstrap.cc:492] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.710497 28736 ts_tablet_manager.cc:1403] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.072s user 0.008s sys 0.000s
I20260320 15:27:40.713465 29643 ts_tablet_manager.cc:1950] T 2ee4a0d559ce4dfea878bd3f1c03c09a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.713313 28736 raft_consensus.cc:359] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.714340 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:40.714010 28736 raft_consensus.cc:385] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.716652 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:40.721171 28736 raft_consensus.cc:740] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.721812 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:40.721817 28736 consensus_queue.cc:260] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.722572 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:40.723513 28736 ts_tablet_manager.cc:1434] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260320 15:27:40.723817 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 130) took 6528 ms. Trace:
I20260320 15:27:40.723937 29643 rpcz_store.cc:276] 0320 15:27:34.195358 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195403 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:40.624102 (+6428699us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:40.624116 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.635343 (+ 11227us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.706288 (+ 70945us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.723793 (+ 17505us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6428404}
I20260320 15:27:40.724080 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:40.724393 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 120) took 6577 ms. Trace:
I20260320 15:27:40.724538 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]) successfully deleted
I20260320 15:27:40.724546 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:40.724881 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:40.725705 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 126) took 6530 ms. Trace:
I20260320 15:27:40.725796 29643 rpcz_store.cc:276] 0320 15:27:34.194804 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195480 (+ 676us) service_pool.cc:225] Handling call
0320 15:27:40.725518 (+6530038us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:40.725529 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.725684 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6529741}
I20260320 15:27:40.726176 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
W20260320 15:27:40.726245 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41da3cb06b1147fbae2695bc5c9ffaa5 already in progress: creating tablet
I20260320 15:27:40.729933 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:40.731748 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:40.732296 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:40.732757 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:40.733490 29640 ts_tablet_manager.cc:1929] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.733940 29640 log.cc:1199] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8a18f32c29224bd883ceb20658dff228
I20260320 15:27:40.743939 28298 raft_consensus.cc:1275] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Refusing update from remote peer d22e73471b9046d28d3cf63c905b11b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260320 15:27:40.745072 29643 tablet_replica.cc:333] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:40.745312 29643 raft_consensus.cc:2243] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: Raft consensus shutting down.
I20260320 15:27:40.745563 29643 pending_rounds.cc:70] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:40.745612 29643 pending_rounds.cc:77] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7266387803837054976 op_type: NO_OP noop_request { }
I20260320 15:27:40.745718 29643 raft_consensus.cc:2889] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:40.745803 29643 raft_consensus.cc:2272] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.746788 29643 ts_tablet_manager.cc:1916] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.747824 29599 consensus_queue.cc:1048] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.724603 29678 rpcz_store.cc:276] 0320 15:27:34.146708 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.146948 (+ 240us) service_pool.cc:225] Handling call
0320 15:27:40.417569 (+6270621us) ts_tablet_manager.cc:1151] Deleting tablet aafd1e123cd043e7828140d2443e047f
0320 15:27:40.417580 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.528141 (+110561us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.673942 (+145801us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.724336 (+ 50394us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6249909}
I20260320 15:27:40.750766 29678 tablet_replica.cc:333] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.751577 29678 raft_consensus.cc:2243] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.752171 29678 raft_consensus.cc:2272] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.752739 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet aafd1e123cd043e7828140d2443e047f (table default.Table_134b3b5c79e344f2bf710b12615479be [id=5186e4c77f9f4f59814887534793f0f0]) successfully deleted
I20260320 15:27:40.759294 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Bootstrap replaying log segment 1/1 (180.4K/241.3K this segment, stats: ops{read=411 overwritten=1 applied=407 ignored=0} inserts{seen=380 ignored=0} mutations{seen=447 ignored=0} orphaned_commits=0)
I20260320 15:27:40.762946 28736 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.767414 28736 tablet_bootstrap.cc:654] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.768618 28733 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.768784 28733 ts_tablet_manager.cc:1403] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.116s user 0.011s sys 0.003s
I20260320 15:27:40.769615 28733 raft_consensus.cc:359] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.769809 28733 raft_consensus.cc:385] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.769867 28733 raft_consensus.cc:740] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.770087 28733 consensus_queue.cc:260] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.772909 29678 ts_tablet_manager.cc:1916] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.775717 29640 ts_tablet_manager.cc:1950] T 8a18f32c29224bd883ceb20658dff228 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:40.779217 28733 ts_tablet_manager.cc:1434] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:40.779582 28733 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.780290 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 117) took 6836 ms. Trace:
I20260320 15:27:40.781536 29640 rpcz_store.cc:276] 0320 15:27:33.943723 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.943783 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:40.261908 (+6318125us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:40.261920 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.330802 (+ 68882us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.733211 (+402409us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.780009 (+ 46798us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6286671}
I20260320 15:27:40.786161 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8a18f32c29224bd883ceb20658dff228 (table default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]) successfully deleted
I20260320 15:27:40.781452 28733 tablet_bootstrap.cc:654] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:40.787288 28383 consensus_peers.cc:597] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6201b74fdb0428e98f2eebb0b24d01a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:40.792373 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:40.793315 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:40.794515 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:40.795374 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:40.796234 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:40.797888 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:40.799103 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:40.799396 30168 raft_consensus.cc:493] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.799585 30168 raft_consensus.cc:515] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.800513 30168 leader_election.cc:290] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.801311 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:40.801682 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:40.802309 28382 leader_election.cc:343] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.802487 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.802736 29640 tablet_replica.cc:333] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:40.803431 28383 leader_election.cc:343] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.803624 28383 leader_election.cc:304] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:40.803469 29640 raft_consensus.cc:2243] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.804168 29640 raft_consensus.cc:2272] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.803351 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:40.804271 30168 raft_consensus.cc:2749] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.804689 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:40.805984 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:40.806016 29640 ts_tablet_manager.cc:1916] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.807467 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:40.810531 28734 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.810731 28734 ts_tablet_manager.cc:1403] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.241s user 0.001s sys 0.005s
I20260320 15:27:40.811650 28734 raft_consensus.cc:359] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.811817 28734 raft_consensus.cc:385] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.811863 28734 raft_consensus.cc:740] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.812078 28734 consensus_queue.cc:260] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.812551 28734 ts_tablet_manager.cc:1434] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:40.812831 28734 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.815124 28734 tablet_bootstrap.cc:654] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.822665 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:40.823583 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:40.824288 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:40.824985 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:40.825853 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:40.826670 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:40.831765 29599 consensus_queue.cc:1048] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:40.833709 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:40.833875 28736 tablet_bootstrap.cc:492] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.834045 28736 ts_tablet_manager.cc:1403] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.071s user 0.002s sys 0.003s
I20260320 15:27:40.834492 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:40.835242 28736 raft_consensus.cc:359] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.835556 28736 raft_consensus.cc:385] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.835649 28736 raft_consensus.cc:740] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.836081 28736 consensus_queue.cc:260] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.837253 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:40.837764 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:40.838209 28736 ts_tablet_manager.cc:1434] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:40.838253 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:40.839084 28736 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:40.839128 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:40.840163 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:40.840772 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:40.846107 30168 raft_consensus.cc:493] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.846285 30168 raft_consensus.cc:515] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.846976 30168 leader_election.cc:290] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:40.847860 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:40.847880 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:40.848426 28383 leader_election.cc:343] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:40.849262 28382 leader_election.cc:343] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:40.849385 28382 leader_election.cc:304] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:40.849867 30168 raft_consensus.cc:2749] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:40.851955 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:40.852878 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:40.853610 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:40.853706 28734 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:40.854261 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:40.854779 28734 ts_tablet_manager.cc:1403] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.000s
I20260320 15:27:40.854722 28890 catalog_manager.cc:5671] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:40.855176 30155 raft_consensus.cc:493] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:40.855350 30155 raft_consensus.cc:515] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.855899 28734 raft_consensus.cc:359] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.856020 30155 leader_election.cc:290] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:40.856627 28890 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:40.856762 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:40.856957 28298 raft_consensus.cc:2468] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:40.857380 28524 leader_election.cc:304] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:40.857714 30155 raft_consensus.cc:2804] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:40.857786 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet e779a274913f4caeaf82e3f180457830
I20260320 15:27:40.857823 30155 raft_consensus.cc:493] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:40.857908 30155 raft_consensus.cc:3060] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.857993 28734 raft_consensus.cc:385] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.858449 28889 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 95) took 3587 ms. Trace:
I20260320 15:27:40.858846 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet e779a274913f4caeaf82e3f180457830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:40.858542 28889 rpcz_store.cc:276] 0320 15:27:37.271433 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.676634 (+3405201us) service_pool.cc:225] Handling call
0320 15:27:40.677300 (+ 666us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:40.677475 (+ 175us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:40.677554 (+ 79us) catalog_manager.cc:2824] Locking tablets
0320 15:27:40.755768 (+ 78214us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:40.762875 (+ 7107us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:40.763005 (+ 130us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:40.857720 (+ 94715us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:40.858376 (+ 656us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.756781 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.756935 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.756944 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:40.756946 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:40.757072 (+ 126us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.757090 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.757095 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:40.757194 (+ 99us) tablet.cc:700] Row locks acquired
0320 15:27:40.757197 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:40.757252 (+ 55us) write_op.cc:270] Start()
0320 15:27:40.757299 (+ 47us) write_op.cc:276] Timestamp: P: 1774020460757225 usec, L: 0
0320 15:27:40.757303 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.757650 (+ 347us) log.cc:844] Serialized 756 byte log entry
0320 15:27:40.762219 (+ 4569us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.762381 (+ 162us) write_op.cc:301] APPLY: starting
0320 15:27:40.762399 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.762447 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.762476 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.762557 (+ 81us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.762594 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:40.762602 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:40.762685 (+ 83us) log.cc:844] Serialized 29 byte log entry
0320 15:27:40.762749 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.762771 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:40.762778 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.762863 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.762867 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":473,"apply.run_wall_time_us":473,"num_ops":2,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":1020,"prepare.run_wall_time_us":1020,"raft.queue_time_us":358,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":318,"replication_time_us":4874}]]}
I20260320 15:27:40.859099 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet e779a274913f4caeaf82e3f180457830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:40.858057 28734 raft_consensus.cc:740] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.860093 28734 consensus_queue.cc:260] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.860749 30155 raft_consensus.cc:515] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.861217 28734 ts_tablet_manager.cc:1434] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:40.861392 30155 leader_election.cc:290] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:40.861953 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet e779a274913f4caeaf82e3f180457830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:40.862461 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:40.862643 28442 raft_consensus.cc:3060] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.863596 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:40.865451 28442 raft_consensus.cc:2468] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.866999 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5618796f1a4ff3ab4c1209dad0aede" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:40.867202 28298 raft_consensus.cc:3060] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:40.870009 28298 raft_consensus.cc:2468] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:40.870538 28524 leader_election.cc:304] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:40.872488 30155 raft_consensus.cc:2804] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:40.872534 29643 ts_tablet_manager.cc:1929] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:40.872641 29643 log.cc:1199] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/757b5afdea674217909610e3719a8804
I20260320 15:27:40.873250 30155 raft_consensus.cc:697] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:40.873188 29643 ts_tablet_manager.cc:1950] T 757b5afdea674217909610e3719a8804 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:40.873653 30155 consensus_queue.cc:237] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:40.875464 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:40.880333 28736 tablet_bootstrap.cc:654] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.885426 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 118) took 6691 ms. Trace:
I20260320 15:27:40.885860 29643 rpcz_store.cc:276] 0320 15:27:34.193792 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195586 (+ 1794us) service_pool.cc:225] Handling call
0320 15:27:40.745007 (+6549421us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:40.745021 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.773032 (+ 28011us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.872408 (+ 99376us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.885393 (+ 12985us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6549082}
I20260320 15:27:40.886540 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 757b5afdea674217909610e3719a8804 (table default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]) successfully deleted
I20260320 15:27:40.894126 29678 ts_tablet_manager.cc:1929] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.894289 29678 log.cc:1199] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/c3bc4dd47817490d88725127b456cc70
I20260320 15:27:40.895323 29678 ts_tablet_manager.cc:1950] T c3bc4dd47817490d88725127b456cc70 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:40.896145 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 129) took 6736 ms. Trace:
I20260320 15:27:40.896354 29678 rpcz_store.cc:276] 0320 15:27:34.160051 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160094 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:40.750548 (+6590454us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:40.750569 (+ 21us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.817547 (+ 66978us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.894003 (+ 76456us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.896127 (+ 2124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6581593}
I20260320 15:27:40.897121 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet c3bc4dd47817490d88725127b456cc70 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:40.897253 28734 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:40.906242 28733 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.906497 29678 tablet_replica.cc:333] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:40.906760 29678 raft_consensus.cc:2243] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:40.906970 29678 raft_consensus.cc:2272] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.907173 28734 tablet_bootstrap.cc:654] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.907634 29678 ts_tablet_manager.cc:1916] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.907648 28733 ts_tablet_manager.cc:1403] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.128s user 0.003s sys 0.018s
I20260320 15:27:40.908670 28733 raft_consensus.cc:359] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.908852 28733 raft_consensus.cc:385] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.908903 28733 raft_consensus.cc:740] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.913987 28733 consensus_queue.cc:260] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.914618 28733 ts_tablet_manager.cc:1434] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260320 15:27:40.914929 28733 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.932168 28733 tablet_bootstrap.cc:654] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.948076 29640 ts_tablet_manager.cc:1929] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:40.948357 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 666ea54f363045038d508d5f291ead5e (DEFAULT_TABLE table=default.Table_f2d3223c396f47d2b377874a6300645d [id=189b496ea69647379380f04c4d9f4166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.948698 29640 log.cc:1199] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7394e941c99e4358ad7a33db84b15348
I20260320 15:27:40.948828 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666ea54f363045038d508d5f291ead5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.952100 28736 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:40.952517 28736 ts_tablet_manager.cc:1403] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.114s user 0.005s sys 0.001s
I20260320 15:27:40.953682 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 666ea54f363045038d508d5f291ead5e (DEFAULT_TABLE table=default.Table_f2d3223c396f47d2b377874a6300645d [id=189b496ea69647379380f04c4d9f4166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.954140 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666ea54f363045038d508d5f291ead5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.954655 28736 raft_consensus.cc:359] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.956043 28736 raft_consensus.cc:385] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.956558 28736 raft_consensus.cc:740] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.958817 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 666ea54f363045038d508d5f291ead5e (DEFAULT_TABLE table=default.Table_f2d3223c396f47d2b377874a6300645d [id=189b496ea69647379380f04c4d9f4166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.958869 29640 ts_tablet_manager.cc:1950] T 7394e941c99e4358ad7a33db84b15348 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:40.963433 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 740981a8159243088548b90b0c576028 (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.964594 28733 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:40.964862 28733 ts_tablet_manager.cc:1403] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.000s
I20260320 15:27:40.959372 28736 consensus_queue.cc:260] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.965931 28733 raft_consensus.cc:359] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.966166 28733 raft_consensus.cc:385] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:40.966259 28733 raft_consensus.cc:740] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:40.966583 28733 consensus_queue.cc:260] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:40.967154 28733 ts_tablet_manager.cc:1434] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:40.970389 28733 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:40.972533 28551 tablet_service.cc:1511] Processing CreateTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.972805 28733 tablet_bootstrap.cc:654] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:40.973131 28413 tablet_service.cc:1511] Processing CreateTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.974062 28277 tablet_service.cc:1511] Processing CreateTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (DEFAULT_TABLE table=default.Table_c66e0ce947a146eeafcac51f268deaa1 [id=a26130d99d8a4f98bb99c3bff5c7017c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.974783 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf37dbb8bc8e48c5a11f106ecd93dd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.976586 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 740981a8159243088548b90b0c576028 (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.977129 28736 ts_tablet_manager.cc:1434] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.004s sys 0.000s
I20260320 15:27:40.978160 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf37dbb8bc8e48c5a11f106ecd93dd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.978706 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 118) took 7032 ms. Trace:
I20260320 15:27:40.978812 29640 rpcz_store.cc:276] 0320 15:27:33.946202 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.946282 (+ 80us) service_pool.cc:225] Handling call
0320 15:27:40.802600 (+6856318us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:40.802655 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.897856 (+ 95201us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.929086 (+ 31230us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:40.978683 (+ 49597us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6826749}
I20260320 15:27:40.979341 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7394e941c99e4358ad7a33db84b15348 (table default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]) successfully deleted
I20260320 15:27:40.979884 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740981a8159243088548b90b0c576028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:40.980157 29640 tablet_replica.cc:333] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:40.981251 29640 raft_consensus.cc:2243] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:40.981705 29640 raft_consensus.cc:2272] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:40.982959 29640 ts_tablet_manager.cc:1916] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:40.987360 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 740981a8159243088548b90b0c576028 (DEFAULT_TABLE table=default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:40.989146 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740981a8159243088548b90b0c576028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:41.001883 28736 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.008899 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf37dbb8bc8e48c5a11f106ecd93dd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:41.011751 29640 ts_tablet_manager.cc:1929] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.012293 29643 tablet_replica.cc:333] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.012351 29640 log.cc:1199] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ac4c7dc6859048ce956927201df44a86
I20260320 15:27:41.012537 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 740981a8159243088548b90b0c576028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:41.013770 29640 ts_tablet_manager.cc:1950] T ac4c7dc6859048ce956927201df44a86 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.012535 29643 raft_consensus.cc:2243] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.015856 29643 raft_consensus.cc:2272] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.016505 29643 ts_tablet_manager.cc:1916] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.016868 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666ea54f363045038d508d5f291ead5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:41.017524 30018 raft_consensus.cc:493] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.017117 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 113) took 7112 ms. Trace:
I20260320 15:27:41.017660 30018 raft_consensus.cc:515] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.017752 29640 rpcz_store.cc:276] 0320 15:27:33.904720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.904770 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:40.979118 (+7074348us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:40.979129 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:40.999118 (+ 19989us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.011221 (+ 12103us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.016995 (+ 5774us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6997106}
I20260320 15:27:41.018397 30018 leader_election.cc:290] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.018936 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ac4c7dc6859048ce956927201df44a86 (table default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]) successfully deleted
I20260320 15:27:41.019263 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.019445 29640 tablet_replica.cc:333] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.019517 28443 raft_consensus.cc:2393] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:41.020213 29640 raft_consensus.cc:2243] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.020862 29640 raft_consensus.cc:2272] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.022182 29640 ts_tablet_manager.cc:1916] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.025341 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd8f5ccbd35418684fd27459c7a6b9f" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.026015 28239 leader_election.cc:304] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:41.026340 30018 raft_consensus.cc:3060] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.029390 30018 raft_consensus.cc:2749] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.037385 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:41.043231 28733 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.043409 28733 ts_tablet_manager.cc:1403] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.073s user 0.003s sys 0.003s
I20260320 15:27:41.044651 28733 raft_consensus.cc:359] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.044850 28733 raft_consensus.cc:385] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.044905 28733 raft_consensus.cc:740] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.045176 28733 consensus_queue.cc:260] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.050081 28733 ts_tablet_manager.cc:1434] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:41.051100 28734 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.051309 28734 ts_tablet_manager.cc:1403] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.154s user 0.002s sys 0.007s
I20260320 15:27:41.052228 28734 raft_consensus.cc:359] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.052502 28734 raft_consensus.cc:385] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.052615 28734 raft_consensus.cc:740] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.052896 28734 consensus_queue.cc:260] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.053035 28736 tablet_bootstrap.cc:654] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:41.053524 28524 consensus_peers.cc:597] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 533105ad6e054fe3a4db32855bc688bd. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:41.054416 28524 consensus_peers.cc:597] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7be03035dc3d44f9aa5b43b49281a940. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.055622 30168 raft_consensus.cc:493] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.055748 30168 raft_consensus.cc:515] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.056391 30168 leader_election.cc:290] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.056496 29640 ts_tablet_manager.cc:1929] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.057032 29640 log.cc:1199] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:41.057192 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:41.057723 28383 leader_election.cc:343] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.057822 28734 ts_tablet_manager.cc:1434] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:41.058112 28734 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.058719 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.059285 28733 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.059329 29640 ts_tablet_manager.cc:1950] T 9a072859f39c4c28ae65b0e9afa79d66 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:41.059636 28382 leader_election.cc:343] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.059746 28382 leader_election.cc:304] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.060403 30169 raft_consensus.cc:2749] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.060530 29643 ts_tablet_manager.cc:1929] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.060783 29643 log.cc:1199] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7394e941c99e4358ad7a33db84b15348
I20260320 15:27:41.061721 29643 ts_tablet_manager.cc:1950] T 7394e941c99e4358ad7a33db84b15348 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.062547 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 112) took 6869 ms. Trace:
I20260320 15:27:41.062844 29643 rpcz_store.cc:276] 0320 15:27:34.192975 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.195972 (+ 2997us) service_pool.cc:225] Handling call
0320 15:27:40.886930 (+6690958us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:41.001253 (+114323us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x61800000c918. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff37d3529ad 00007ff37d353a1e 00007ff390fe279d 00007ff390f614c3 00007ff38d17ce26 00007ff38d17e1b9 00007ff38d1b1b44 00007ff38d192560 00007ff38d1922ed 0000000000a8b58c
0320 15:27:41.012215 (+ 10962us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.020449 (+ 8234us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.031983 (+ 11534us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.062487 (+ 30504us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":260992768,"tablet-delete.queue_time_us":6690726}
I20260320 15:27:41.063459 28734 tablet_bootstrap.cc:654] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.063843 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7394e941c99e4358ad7a33db84b15348 (table default.Table_2b163b340195441daf46b7d424a4ebdf [id=0ff86d45b3eb4a128ec94121f4edb5bb]) successfully deleted
I20260320 15:27:41.065634 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 114) took 7157 ms. Trace:
I20260320 15:27:41.066262 29640 rpcz_store.cc:276] 0320 15:27:33.907816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.907882 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:41.019192 (+7111310us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:41.019208 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.030370 (+ 11162us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.055984 (+ 25614us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.065434 (+ 9450us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7034555}
I20260320 15:27:41.068801 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9a072859f39c4c28ae65b0e9afa79d66 (table default.Table_874d6a667bff4c3d815fcff90cae13f5 [id=aa8e982cd47c499a90f51cad2f96548e]) successfully deleted
I20260320 15:27:41.069505 29640 tablet_replica.cc:333] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:41.070698 28524 consensus_peers.cc:597] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 533105ad6e054fe3a4db32855bc688bd. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.070856 29939 consensus_queue.cc:1048] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.070950 29640 raft_consensus.cc:2243] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.071786 29640 raft_consensus.cc:2272] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.073140 28733 tablet_bootstrap.cc:654] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.076334 30018 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:41.076524 30018 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.076615 29640 ts_tablet_manager.cc:1916] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.077150 30018 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:41.078838 28239 leader_election.cc:343] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:41.080776 30168 raft_consensus.cc:493] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.080933 30168 raft_consensus.cc:515] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.081565 30168 leader_election.cc:290] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.082381 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.082602 28584 raft_consensus.cc:2468] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:41.083216 28382 leader_election.cc:304] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.083696 30168 raft_consensus.cc:2804] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.083808 30168 raft_consensus.cc:493] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.083890 30168 raft_consensus.cc:3060] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.086992 30168 raft_consensus.cc:515] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.087469 29678 ts_tablet_manager.cc:1929] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.087869 29678 log.cc:1199] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7feb8477ebf841a19977a434f365f537
I20260320 15:27:41.088421 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.088757 29678 ts_tablet_manager.cc:1950] T 7feb8477ebf841a19977a434f365f537 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:41.089003 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.089232 28584 raft_consensus.cc:3060] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.092082 30168 leader_election.cc:290] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.092849 28584 raft_consensus.cc:2468] T e0e7ae1ff302428abf6cc592bbca0aa9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:41.093581 28382 leader_election.cc:304] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.093575 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 136) took 6925 ms. Trace:
W20260320 15:27:41.094918 28383 leader_election.cc:343] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.064572 29643 tablet_replica.cc:333] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.097249 30018 raft_consensus.cc:493] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.097414 30018 raft_consensus.cc:515] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.098026 30018 leader_election.cc:290] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.098969 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:41.099965 28574 raft_consensus.cc:3060] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.101444 30170 raft_consensus.cc:2804] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.101603 30170 raft_consensus.cc:697] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:41.101884 30170 consensus_queue.cc:237] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.102490 29643 raft_consensus.cc:2243] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.102862 29643 raft_consensus.cc:2272] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.103483 29643 ts_tablet_manager.cc:1916] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.107417 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:41.108212 28383 leader_election.cc:343] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:41.109644 28239 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.109916 30018 consensus_queue.cc:1048] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.114005 29640 ts_tablet_manager.cc:1929] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.114671 29640 log.cc:1199] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:41.115420 29599 consensus_queue.cc:1048] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.116637 29640 ts_tablet_manager.cc:1950] T ea95b7a1b30e42cebcbfcd551c58f679 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:41.122895 28524 consensus_peers.cc:597] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3bc4dd47817490d88725127b456cc70. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.123711 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 111) took 7227 ms. Trace:
I20260320 15:27:41.124056 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
W20260320 15:27:41.123515 28239 consensus_peers.cc:597] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.126931 29640 rpcz_store.cc:276] 0320 15:27:33.896593 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.896669 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:41.068329 (+7171660us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:41.069284 (+ 955us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.085343 (+ 16059us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.113599 (+ 28256us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.123586 (+ 9987us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7082369}
I20260320 15:27:41.128698 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ea95b7a1b30e42cebcbfcd551c58f679 (table default.Table_0703f70973474e499d7916aa2a63f195 [id=f3981620a2d444448c0c93103cc27a97]) successfully deleted
I20260320 15:27:41.129531 29640 tablet_replica.cc:333] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.130677 29640 raft_consensus.cc:2243] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.131582 29640 raft_consensus.cc:2272] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.131955 29643 ts_tablet_manager.cc:1929] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.133651 29643 log.cc:1199] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b6201b74fdb0428e98f2eebb0b24d01a
I20260320 15:27:41.134382 29643 ts_tablet_manager.cc:1950] T b6201b74fdb0428e98f2eebb0b24d01a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:41.134742 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6bbc5d5f49a04216a352ddb1b571dce0 consensus state has no leader
I20260320 15:27:41.135859 29640 ts_tablet_manager.cc:1916] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.137087 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 113) took 6943 ms. Trace:
I20260320 15:27:41.146250 29643 rpcz_store.cc:276] 0320 15:27:34.193154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.196287 (+ 3133us) service_pool.cc:225] Handling call
0320 15:27:41.064317 (+6868030us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:41.064332 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.113438 (+ 49106us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.131831 (+ 18393us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.137065 (+ 5234us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6867780}
I20260320 15:27:41.156113 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b6201b74fdb0428e98f2eebb0b24d01a (table default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]) successfully deleted
I20260320 15:27:41.156795 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 114) took 6963 ms. Trace:
I20260320 15:27:41.156956 29643 rpcz_store.cc:276] 0320 15:27:34.193295 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.196527 (+ 3232us) service_pool.cc:225] Handling call
0320 15:27:41.156563 (+6960036us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:41.156628 (+ 65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.156770 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6959865}
W20260320 15:27:41.157647 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6292904782fd4b619483135e58b66684 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6292904782fd4b619483135e58b66684 already in progress: creating tablet
I20260320 15:27:41.158106 29643 tablet_replica.cc:333] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.158605 29643 raft_consensus.cc:2243] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.158895 29643 raft_consensus.cc:2272] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.159713 29643 ts_tablet_manager.cc:1916] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.165740 28736 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.166028 28736 ts_tablet_manager.cc:1403] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.166s user 0.004s sys 0.004s
I20260320 15:27:41.168193 28736 raft_consensus.cc:359] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.168699 28736 raft_consensus.cc:385] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.168889 28736 raft_consensus.cc:740] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.093879 29678 rpcz_store.cc:276] 0320 15:27:34.167955 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.168009 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:40.897165 (+6729156us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:40.897213 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.035094 (+137881us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.087137 (+ 52043us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.093509 (+ 6372us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6722575}
I20260320 15:27:41.169994 28734 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.170329 28734 ts_tablet_manager.cc:1403] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.112s user 0.007s sys 0.009s
I20260320 15:27:41.170756 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 130) took 7010 ms. Trace:
I20260320 15:27:41.170966 29678 rpcz_store.cc:276] 0320 15:27:34.160231 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160340 (+ 109us) service_pool.cc:225] Handling call
0320 15:27:41.170363 (+7010023us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:41.170407 (+ 44us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.170698 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6993865}
I20260320 15:27:41.171216 28734 raft_consensus.cc:359] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.171569 28734 raft_consensus.cc:385] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.171677 28734 raft_consensus.cc:740] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.171972 28734 consensus_queue.cc:260] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.171994 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 125) took 7012 ms. Trace:
I20260320 15:27:41.172192 29678 rpcz_store.cc:276] 0320 15:27:34.159092 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160576 (+ 1484us) service_pool.cc:225] Handling call
0320 15:27:41.171598 (+7011022us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:41.171649 (+ 51us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.171938 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6994759}
I20260320 15:27:41.172880 28734 ts_tablet_manager.cc:1434] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:41.173259 28734 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.175158 28734 tablet_bootstrap.cc:654] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.178514 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7feb8477ebf841a19977a434f365f537 (table default.Table_fdd90d848d4846dcb37f6b50bce8156b [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
W20260320 15:27:41.178877 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8241a342054240d2ac62b729cc9c9a62 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8241a342054240d2ac62b729cc9c9a62 already in progress: creating tablet
I20260320 15:27:41.178885 28733 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.179221 28733 ts_tablet_manager.cc:1403] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.021s
I20260320 15:27:41.181231 28733 raft_consensus.cc:359] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.182087 28733 raft_consensus.cc:385] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.169461 28736 consensus_queue.cc:260] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.183770 29643 ts_tablet_manager.cc:1929] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.183123 28733 raft_consensus.cc:740] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.184711 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.186437 28440 raft_consensus.cc:2468] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.186842 30018 raft_consensus.cc:493] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.186960 30018 raft_consensus.cc:515] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.187258 29643 log.cc:1199] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/3e7b964395f24e45bc2715be3fac6dbe
I20260320 15:27:41.187606 30018 leader_election.cc:290] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.187794 29643 ts_tablet_manager.cc:1950] T 3e7b964395f24e45bc2715be3fac6dbe P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.185510 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.188040 28443 raft_consensus.cc:2468] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.186211 28733 consensus_queue.cc:260] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.188352 28440 raft_consensus.cc:1275] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Refusing update from remote peer 3ce5383d28b347ab979765b880e5773c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260320 15:27:41.189328 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 116) took 6995 ms. Trace:
I20260320 15:27:41.189991 29678 tablet_replica.cc:333] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:41.190214 29678 raft_consensus.cc:2243] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.189783 28736 ts_tablet_manager.cc:1434] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260320 15:27:41.192250 28240 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.193188 30018 raft_consensus.cc:2804] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:41.193369 30018 raft_consensus.cc:493] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:41.193481 30018 raft_consensus.cc:3060] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:41.193099 29643 rpcz_store.cc:276] 0320 15:27:34.193555 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.197155 (+ 3600us) service_pool.cc:225] Handling call
0320 15:27:41.158049 (+6960894us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:41.158063 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.164073 (+ 6010us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.183651 (+ 19578us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.189310 (+ 5659us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6960688}
I20260320 15:27:41.194811 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 3e7b964395f24e45bc2715be3fac6dbe (table default.Table_c783052b0eb045ea95d77f3722e1ca8b [id=eba1ff6618e0486898840ab0fd2f0882]) successfully deleted
I20260320 15:27:41.194748 28736 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.195416 29643 tablet_replica.cc:333] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.194825 28733 ts_tablet_manager.cc:1434] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.015s user 0.000s sys 0.003s
I20260320 15:27:41.196063 29643 raft_consensus.cc:2243] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.196552 29643 raft_consensus.cc:2272] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.190529 29678 raft_consensus.cc:2272] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.197095 30018 raft_consensus.cc:515] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.197849 29643 ts_tablet_manager.cc:1916] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.198151 30018 leader_election.cc:290] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.191084 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.198495 28440 raft_consensus.cc:2468] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.200682 28733 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.203907 28240 leader_election.cc:304] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.205379 30018 raft_consensus.cc:2804] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.205520 30018 raft_consensus.cc:493] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.205600 30018 raft_consensus.cc:3060] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.206337 29678 ts_tablet_manager.cc:1916] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.209731 30018 raft_consensus.cc:515] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.209888 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.210182 28584 raft_consensus.cc:2468] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.211473 30114 consensus_queue.cc:1048] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260320 15:27:41.210603 30018 leader_election.cc:290] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.212657 30114 consensus_queue.cc:1048] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.218372 30170 raft_consensus.cc:493] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.218648 30170 raft_consensus.cc:515] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.219609 30170 leader_election.cc:290] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.220670 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fa691a26114657ad768545a30dfd3c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.221196 28240 leader_election.cc:304] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:41.221349 28382 leader_election.cc:343] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.221681 30114 raft_consensus.cc:2804] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.221649 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.221778 30114 raft_consensus.cc:493] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.221858 30114 raft_consensus.cc:3060] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.222332 29643 ts_tablet_manager.cc:1929] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.222445 29643 log.cc:1199] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:41.222790 29643 ts_tablet_manager.cc:1950] T 1bf5e17dea5445d9909a229fa8315348 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.223591 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 119) took 7029 ms. Trace:
I20260320 15:27:41.223688 29643 rpcz_store.cc:276] 0320 15:27:34.193983 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.197393 (+ 3410us) service_pool.cc:225] Handling call
0320 15:27:41.195356 (+6997963us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:41.195369 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.207959 (+ 12590us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.222201 (+ 14242us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.223567 (+ 1366us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6997801}
I20260320 15:27:41.224193 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1bf5e17dea5445d9909a229fa8315348 (table default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]) successfully deleted
I20260320 15:27:41.224648 30114 raft_consensus.cc:515] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.225353 30114 leader_election.cc:290] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.227567 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5438fdc6e6944e79ac8aecd0ef774467" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.227766 28440 raft_consensus.cc:3060] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:41.230567 28440 raft_consensus.cc:2468] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 2.
I20260320 15:27:41.229957 28736 tablet_bootstrap.cc:654] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.230315 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:41.232077 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:41.227880 28239 leader_election.cc:343] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:41.232610 28440 raft_consensus.cc:3060] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.233148 29643 tablet_replica.cc:333] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:41.233180 28239 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.233362 29643 raft_consensus.cc:2243] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.233099 28733 tablet_bootstrap.cc:654] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:41.234373 30168 consensus_queue.cc:1175] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "80636bb608a1431b955576e296ff63bb" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260320 15:27:41.234668 29643 raft_consensus.cc:2272] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.235275 29643 ts_tablet_manager.cc:1916] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.237214 28440 raft_consensus.cc:2468] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.235994 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
W20260320 15:27:41.239727 28524 consensus_peers.cc:597] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bf5e17dea5445d9909a229fa8315348. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.242600 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:41.246491 29640 ts_tablet_manager.cc:1929] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.246783 29640 log.cc:1199] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:41.247733 29640 ts_tablet_manager.cc:1950] T b2355eadcae6483bae7ae475d74dd131 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.248451 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.248764 28584 raft_consensus.cc:3060] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.249558 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 119) took 7301 ms. Trace:
I20260320 15:27:41.249743 29640 rpcz_store.cc:276] 0320 15:27:33.948041 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.948127 (+ 86us) service_pool.cc:225] Handling call
0320 15:27:41.129147 (+7181020us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:41.129222 (+ 75us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.191443 (+ 62221us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.246210 (+ 54767us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.249536 (+ 3326us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7141161}
I20260320 15:27:41.250541 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b2355eadcae6483bae7ae475d74dd131 (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:41.250877 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23eda54577404cc6900cb3ae348604d4" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.251147 28445 raft_consensus.cc:3060] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.253610 29643 ts_tablet_manager.cc:1929] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.253695 28445 raft_consensus.cc:2468] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.253786 28584 raft_consensus.cc:2468] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.254653 28240 leader_election.cc:304] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.255595 30114 raft_consensus.cc:2804] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:41.255981 30114 raft_consensus.cc:697] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.253736 29643 log.cc:1199] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:41.256364 30114 consensus_queue.cc:237] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.257119 29643 ts_tablet_manager.cc:1950] T ea6ff59d2ffe4d579e6db2375eb11414 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.258424 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 121) took 7064 ms. Trace:
I20260320 15:27:41.250437 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:41.259984 29643 rpcz_store.cc:276] 0320 15:27:34.194193 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.197581 (+ 3388us) service_pool.cc:225] Handling call
0320 15:27:41.233089 (+7035508us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:41.233105 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.238480 (+ 5375us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.253471 (+ 14991us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.258393 (+ 4922us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":232192,"tablet-delete.queue_time_us":7035297}
I20260320 15:27:41.260320 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:41.261127 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ea6ff59d2ffe4d579e6db2375eb11414 (table default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]) successfully deleted
I20260320 15:27:41.261195 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:41.261816 29643 tablet_replica.cc:333] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.262403 29643 raft_consensus.cc:2243] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.262708 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:41.263059 29643 raft_consensus.cc:2272] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.263397 28240 leader_election.cc:304] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.264156 29643 ts_tablet_manager.cc:1916] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.264424 30018 raft_consensus.cc:2804] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.264781 30018 raft_consensus.cc:697] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.265103 30018 consensus_queue.cc:237] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.266085 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fa691a26114657ad768545a30dfd3c" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.268703 28240 leader_election.cc:304] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.269048 29618 raft_consensus.cc:2804] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.271469 28733 tablet_bootstrap.cc:492] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.271803 28733 ts_tablet_manager.cc:1403] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.007s
I20260320 15:27:41.272626 29640 tablet_replica.cc:333] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.273100 29640 raft_consensus.cc:2243] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.273510 29640 raft_consensus.cc:2272] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.275520 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:41.281737 29643 ts_tablet_manager.cc:1929] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.282778 28734 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.283092 28734 ts_tablet_manager.cc:1403] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.110s user 0.022s sys 0.011s
I20260320 15:27:41.283608 29643 log.cc:1199] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:41.284072 29643 ts_tablet_manager.cc:1950] T 5fa5f6fe801b4452bf294a9ae23bb163 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.285262 28736 tablet_bootstrap.cc:492] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.285442 28736 ts_tablet_manager.cc:1403] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.092s user 0.005s sys 0.004s
I20260320 15:27:41.286337 28736 raft_consensus.cc:359] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.286541 28736 raft_consensus.cc:385] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.286605 28736 raft_consensus.cc:740] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.286825 28736 consensus_queue.cc:260] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.287360 28736 ts_tablet_manager.cc:1434] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:41.277014 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:41.288556 29640 ts_tablet_manager.cc:1916] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.284448 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 122) took 7090 ms. Trace:
I20260320 15:27:41.284242 28734 raft_consensus.cc:359] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.290449 28734 raft_consensus.cc:385] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.290596 28734 raft_consensus.cc:740] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.290916 28734 consensus_queue.cc:260] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.290084 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:41.291779 28734 ts_tablet_manager.cc:1434] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:41.292196 28734 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.290169 28733 raft_consensus.cc:359] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.292917 28733 raft_consensus.cc:385] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.294209 30170 raft_consensus.cc:493] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.294368 30170 raft_consensus.cc:515] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.294312 28733 raft_consensus.cc:740] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.295208 28734 tablet_bootstrap.cc:654] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.296101 29643 rpcz_store.cc:276] 0320 15:27:34.194316 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.197814 (+ 3498us) service_pool.cc:225] Handling call
0320 15:27:41.261737 (+7063923us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:41.261755 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.270448 (+ 8693us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.274630 (+ 4182us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.284430 (+ 9800us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7063738}
I20260320 15:27:41.296558 30170 leader_election.cc:290] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.297462 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:41.297537 29643 tablet_replica.cc:333] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.296458 28733 consensus_queue.cc:260] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.298005 29643 raft_consensus.cc:2243] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.298183 29643 raft_consensus.cc:2272] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.299012 29643 ts_tablet_manager.cc:1916] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.299499 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:41.300375 28382 leader_election.cc:343] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.301481 29618 raft_consensus.cc:697] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.301868 29618 consensus_queue.cc:237] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.303980 28733 ts_tablet_manager.cc:1434] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.032s user 0.000s sys 0.004s
I20260320 15:27:41.303840 29678 ts_tablet_manager.cc:1929] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.304971 29678 log.cc:1199] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8a18f32c29224bd883ceb20658dff228
I20260320 15:27:41.307686 28736 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.308548 29643 ts_tablet_manager.cc:1929] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:41.308668 29643 log.cc:1199] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/889d620c311a42039c22af53ad0802f2
I20260320 15:27:41.308971 29643 ts_tablet_manager.cc:1950] T 889d620c311a42039c22af53ad0802f2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.309324 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 124) took 7114 ms. Trace:
I20260320 15:27:41.309434 29643 rpcz_store.cc:276] 0320 15:27:34.194553 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.198029 (+ 3476us) service_pool.cc:225] Handling call
0320 15:27:41.297380 (+7099351us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:41.297396 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.304163 (+ 6767us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.308422 (+ 4259us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.309300 (+ 878us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7099160}
I20260320 15:27:41.309628 28736 tablet_bootstrap.cc:654] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.307341 28733 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:41.313508 28383 leader_election.cc:343] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.313565 29599 consensus_queue.cc:1048] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.313711 28383 leader_election.cc:304] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.313637 29678 ts_tablet_manager.cc:1950] T 8a18f32c29224bd883ceb20658dff228 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:41.314822 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.315099 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:41.315232 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 5fa5f6fe801b4452bf294a9ae23bb163 (table default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]) successfully deleted
I20260320 15:27:41.316108 29643 tablet_replica.cc:333] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.316365 29643 raft_consensus.cc:2243] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.316459 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:41.316860 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:41.316896 30170 raft_consensus.cc:2749] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.317206 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:41.317589 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:41.317781 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:41.318020 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:41.318192 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
W20260320 15:27:41.318317 28383 leader_election.cc:343] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.318475 28383 leader_election.cc:304] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.318544 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:41.318739 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:41.319097 30168 raft_consensus.cc:2749] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.319106 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:41.319171 28295 raft_consensus.cc:3060] T 60d0b36f462945cbbca7e389edf494d3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.319356 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:41.324261 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 131) took 7163 ms. Trace:
I20260320 15:27:41.324486 29678 rpcz_store.cc:276] 0320 15:27:34.160517 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160639 (+ 122us) service_pool.cc:225] Handling call
0320 15:27:41.172685 (+7012046us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:41.189918 (+ 17233us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x61800000c918. stack: 00007f7f40429076 00007f7f40428e30 00007f7f3f3e33ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f7f404119ad 00007f7f40412a1e 00007f7f540a179d 00007f7f540204c3 00007f7f5023be26 00007f7f5023d1b9 00007f7f50270b44 00007f7f50251560 00007f7f502512ed 0000000000a8b58c
0320 15:27:41.189936 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.257659 (+ 67723us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.302378 (+ 44719us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.324199 (+ 21821us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":38966144,"tablet-delete.queue_time_us":6995669}
I20260320 15:27:41.325416 30153 consensus_queue.cc:1048] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.326803 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8a18f32c29224bd883ceb20658dff228 (table default.Table_f40bc6a4b2834fcc860499b3353f1bff [id=b3e91b98938c41d69ab25133d7a650c6]) successfully deleted
I20260320 15:27:41.325338 29678 tablet_replica.cc:333] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:41.334550 28736 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.334723 28736 ts_tablet_manager.cc:1403] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.001s
I20260320 15:27:41.336396 28736 raft_consensus.cc:359] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.336813 28736 raft_consensus.cc:385] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.336951 28736 raft_consensus.cc:740] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.337371 28736 consensus_queue.cc:260] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.338479 28736 ts_tablet_manager.cc:1434] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260320 15:27:41.339134 28736 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.342686 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:41.343271 28889 catalog_manager.cc:5671] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:41.345715 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:41.347966 29678 raft_consensus.cc:2243] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.349434 28733 tablet_bootstrap.cc:654] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.349864 29678 raft_consensus.cc:2272] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.356979 29678 ts_tablet_manager.cc:1916] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.317358 29643 raft_consensus.cc:2272] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.359091 29643 ts_tablet_manager.cc:1916] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.363997 30153 consensus_queue.cc:1048] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.365198 30153 consensus_queue.cc:1048] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.362644 28889 catalog_manager.cc:5671] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:41.367462 28736 tablet_bootstrap.cc:654] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.371807 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:41.372234 29618 raft_consensus.cc:493] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.372459 29618 raft_consensus.cc:515] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.372812 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:41.373651 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.373853 28445 raft_consensus.cc:2468] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.374269 28240 leader_election.cc:304] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.374779 30257 raft_consensus.cc:2804] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.374861 30257 raft_consensus.cc:493] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.374924 30257 raft_consensus.cc:3060] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.377736 30257 raft_consensus.cc:515] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.383278 29640 ts_tablet_manager.cc:1929] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.383201 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:41.385809 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.386015 28445 raft_consensus.cc:3060] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.385969 29640 log.cc:1199] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:41.389001 28445 raft_consensus.cc:2468] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.391327 28240 leader_election.cc:304] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.391772 30256 raft_consensus.cc:2804] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.391911 30256 raft_consensus.cc:697] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.392153 30256 consensus_queue.cc:237] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.394469 28736 tablet_bootstrap.cc:492] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.392905 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:41.396540 29640 ts_tablet_manager.cc:1950] T 26818e620ff649d18b671e4f4c7c7e74 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.396518 30257 leader_election.cc:290] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.397668 28736 ts_tablet_manager.cc:1403] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.059s user 0.000s sys 0.006s
I20260320 15:27:41.399917 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.400123 28574 raft_consensus.cc:3060] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.403080 28574 raft_consensus.cc:2468] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.403825 30153 raft_consensus.cc:493] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:41.403972 30153 raft_consensus.cc:515] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.405175 30153 leader_election.cc:290] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.403726 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
W20260320 15:27:41.406597 28524 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
W20260320 15:27:41.409102 28524 leader_election.cc:343] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:41.409310 28524 leader_election.cc:304] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:41.409852 30153 raft_consensus.cc:2749] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:41.410351 29618 leader_election.cc:290] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:41.412230 28383 consensus_peers.cc:597] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7feb8477ebf841a19977a434f365f537. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.412920 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 120) took 7413 ms. Trace:
I20260320 15:27:41.414319 29640 rpcz_store.cc:276] 0320 15:27:33.999245 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:33.999311 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:41.272491 (+7273180us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:41.272543 (+ 52us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.328022 (+ 55479us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.380773 (+ 52751us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.412489 (+ 31716us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7271253}
I20260320 15:27:41.416821 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:41.415875 28736 raft_consensus.cc:359] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.420323 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 26818e620ff649d18b671e4f4c7c7e74 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
I20260320 15:27:41.419180 28736 raft_consensus.cc:385] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.421384 28736 raft_consensus.cc:740] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.422313 29640 tablet_replica.cc:333] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.423825 29643 ts_tablet_manager.cc:1929] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.424082 29643 log.cc:1199] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/c3bc4dd47817490d88725127b456cc70
I20260320 15:27:41.424587 29618 raft_consensus.cc:493] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.424710 29643 ts_tablet_manager.cc:1950] T c3bc4dd47817490d88725127b456cc70 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.424777 29618 raft_consensus.cc:515] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.425892 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 129) took 7230 ms. Trace:
I20260320 15:27:41.426162 29618 leader_election.cc:290] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.426016 29643 rpcz_store.cc:276] 0320 15:27:34.195207 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.198256 (+ 3049us) service_pool.cc:225] Handling call
0320 15:27:41.316033 (+7117777us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:41.316053 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.389739 (+ 73686us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.423641 (+ 33902us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.425862 (+ 2221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7117594}
I20260320 15:27:41.426606 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 131) took 7231 ms. Trace:
I20260320 15:27:41.426625 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet c3bc4dd47817490d88725127b456cc70 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:41.426694 29643 rpcz_store.cc:276] 0320 15:27:34.195515 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.203793 (+ 8278us) service_pool.cc:225] Handling call
0320 15:27:41.426435 (+7222642us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:41.426448 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.426587 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7222276}
I20260320 15:27:41.426977 29643 tablet_replica.cc:333] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.427217 29643 raft_consensus.cc:2243] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.427421 29643 raft_consensus.cc:2272] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:41.427573 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6d9b0f1936f54c699211c5ee9709f9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d9b0f1936f54c699211c5ee9709f9ed already in progress: creating tablet
I20260320 15:27:41.426980 29640 raft_consensus.cc:2243] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.427735 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.428308 28443 raft_consensus.cc:2468] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.427580 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:41.429582 29640 raft_consensus.cc:2272] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.435642 28900 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 35347116811e4aee8142a503dd21843a
I20260320 15:27:41.436318 28900 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 89) took 5561 ms. Trace:
I20260320 15:27:41.436774 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 35347116811e4aee8142a503dd21843a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.436424 28900 rpcz_store.cc:276] 0320 15:27:35.874579 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.435765 (+4561186us) service_pool.cc:225] Handling call
0320 15:27:40.456361 (+ 20596us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:40.456529 (+ 168us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:40.456615 (+ 86us) catalog_manager.cc:2824] Locking tablets
0320 15:27:40.456772 (+ 157us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:40.463835 (+ 7063us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.463843 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:40.464204 (+ 361us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:40.464314 (+ 110us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:41.435565 (+971251us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:41.436238 (+ 673us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.457769 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.457936 (+ 167us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.457946 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:40.457950 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:40.458107 (+ 157us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.458124 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.458130 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:40.458247 (+ 117us) tablet.cc:700] Row locks acquired
0320 15:27:40.458249 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:40.458298 (+ 49us) write_op.cc:270] Start()
0320 15:27:40.458337 (+ 39us) write_op.cc:276] Timestamp: P: 1774020460458248 usec, L: 0
0320 15:27:40.458341 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.458578 (+ 237us) log.cc:844] Serialized 765 byte log entry
0320 15:27:40.462614 (+ 4036us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.462734 (+ 120us) write_op.cc:301] APPLY: starting
0320 15:27:40.462756 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.462811 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.462842 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.462921 (+ 79us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.462957 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:40.462964 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:40.463055 (+ 91us) log.cc:844] Serialized 29 byte log entry
0320 15:27:40.463130 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.463155 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:40.463164 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":63,"apply.run_cpu_time_us":496,"apply.run_wall_time_us":496,"num_ops":2,"prepare.queue_time_us":174,"prepare.run_cpu_time_us":923,"prepare.run_wall_time_us":924,"raft.queue_time_us":49,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":242,"replication_time_us":4238}]]}
I20260320 15:27:41.428133 29643 ts_tablet_manager.cc:1916] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.446079 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 35347116811e4aee8142a503dd21843a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:41.450366 28734 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.450712 28734 ts_tablet_manager.cc:1403] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.159s user 0.042s sys 0.052s
I20260320 15:27:41.451591 29640 ts_tablet_manager.cc:1916] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.452360 28734 raft_consensus.cc:359] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.452764 28734 raft_consensus.cc:385] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.452916 28734 raft_consensus.cc:740] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.453367 28734 consensus_queue.cc:260] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.454015 29643 ts_tablet_manager.cc:1929] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.454216 29643 log.cc:1199] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9176e734212849aab8d99473de02cc79
I20260320 15:27:41.454725 29643 ts_tablet_manager.cc:1950] T 9176e734212849aab8d99473de02cc79 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.454677 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07af28f257d745a1bb4173aad09045d9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.454898 28574 raft_consensus.cc:2376] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1. Re-sending same reply.
I20260320 15:27:41.455868 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.465330 30153 raft_consensus.cc:493] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.465499 30153 raft_consensus.cc:515] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.466225 30153 leader_election.cc:290] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.467161 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.467415 28443 raft_consensus.cc:2468] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:41.468068 28524 leader_election.cc:304] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.468468 30153 raft_consensus.cc:2804] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.468611 30153 raft_consensus.cc:493] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.468739 30153 raft_consensus.cc:3060] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.426241 28736 consensus_queue.cc:260] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.428762 28240 leader_election.cc:304] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.470721 28736 ts_tablet_manager.cc:1434] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.072s user 0.000s sys 0.004s
I20260320 15:27:41.470875 29618 raft_consensus.cc:2804] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.470942 29618 raft_consensus.cc:493] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.471002 29618 raft_consensus.cc:3060] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.471036 28736 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.464396 30256 raft_consensus.cc:493] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.471357 30256 raft_consensus.cc:515] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.472023 30256 leader_election.cc:290] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.472893 28736 tablet_bootstrap.cc:654] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.474079 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.474509 28445 raft_consensus.cc:2468] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.474946 28240 leader_election.cc:304] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.475301 30256 raft_consensus.cc:2804] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.475373 30256 raft_consensus.cc:493] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.475426 30256 raft_consensus.cc:3060] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.469089 29678 ts_tablet_manager.cc:1929] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.477346 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:41.478384 30256 raft_consensus.cc:515] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:41.481843 28901 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52ab0f33ad6b4cdfbe36e707872bee95 consensus state has no leader
I20260320 15:27:41.483044 28901 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 93) took 4260 ms. Trace:
I20260320 15:27:41.483213 28901 rpcz_store.cc:276] 0320 15:27:37.221645 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.469345 (+3247700us) service_pool.cc:225] Handling call
0320 15:27:40.508155 (+ 38810us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:40.509013 (+ 858us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:40.509213 (+ 200us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:40.509309 (+ 96us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:40.520546 (+ 11237us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:41.482036 (+961490us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.509940 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.510046 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.510055 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:40.510058 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:40.510169 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.510186 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.510191 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:40.510268 (+ 77us) tablet.cc:700] Row locks acquired
0320 15:27:40.510272 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:40.510314 (+ 42us) write_op.cc:270] Start()
0320 15:27:40.510354 (+ 40us) write_op.cc:276] Timestamp: P: 1774020460510265 usec, L: 0
0320 15:27:40.510359 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.510568 (+ 209us) log.cc:844] Serialized 405 byte log entry
0320 15:27:40.518494 (+ 7926us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.519007 (+ 513us) write_op.cc:301] APPLY: starting
0320 15:27:40.519037 (+ 30us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.519098 (+ 61us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.519142 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.519220 (+ 78us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.519283 (+ 63us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:40.519293 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:40.519405 (+ 112us) log.cc:844] Serialized 23 byte log entry
0320 15:27:40.519488 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.519517 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:40.519546 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.519639 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.519645 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":402,"apply.run_cpu_time_us":627,"apply.run_wall_time_us":627,"num_ops":1,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":749,"prepare.run_wall_time_us":748,"raft.queue_time_us":257,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":212,"replication_time_us":8086}]]}
I20260320 15:27:41.484110 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 133) took 7280 ms. Trace:
I20260320 15:27:41.484203 29643 rpcz_store.cc:276] 0320 15:27:34.203846 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.203986 (+ 140us) service_pool.cc:225] Handling call
0320 15:27:41.426915 (+7222929us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:41.426928 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.450621 (+ 23693us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.453905 (+ 3284us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.484086 (+ 30181us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28283,"tablet-delete.queue_time_us":7222657}
I20260320 15:27:41.484535 29643 tablet_replica.cc:333] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.484723 29643 raft_consensus.cc:2243] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.484885 29643 raft_consensus.cc:2272] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.485002 28734 ts_tablet_manager.cc:1434] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260320 15:27:41.485368 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 87cb35cefee841d0881b08c456df5a97
I20260320 15:27:41.485394 29643 ts_tablet_manager.cc:1916] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.486860 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9176e734212849aab8d99473de02cc79 (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:41.485862 28734 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.489658 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.489995 28445 raft_consensus.cc:3060] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.492244 30256 leader_election.cc:290] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.492643 29643 ts_tablet_manager.cc:1929] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:41.477564 28239 leader_election.cc:343] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.484907 29618 raft_consensus.cc:515] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.493305 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 35347116811e4aee8142a503dd21843a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:41.494182 29618 leader_election.cc:290] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.495605 29599 raft_consensus.cc:493] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:41.495766 29599 raft_consensus.cc:515] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.496443 29599 leader_election.cc:290] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.496410 28583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.496593 28583 raft_consensus.cc:2468] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:41.496830 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d78ab2f7e0d7413ea9049dba9cbd8633" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.496903 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:41.496977 28576 raft_consensus.cc:3060] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:41.500046 28524 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:41.500350 28576 raft_consensus.cc:2468] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.500944 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.501137 28295 raft_consensus.cc:2468] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:41.502197 28239 leader_election.cc:343] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.503829 28239 leader_election.cc:304] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:41.504062 28902 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 94) took 4282 ms. Trace:
I20260320 15:27:41.504102 29618 raft_consensus.cc:2804] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.504228 29618 raft_consensus.cc:697] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.504485 29618 consensus_queue.cc:237] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.504611 29599 raft_consensus.cc:493] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.504753 29599 raft_consensus.cc:515] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.504802 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 87cb35cefee841d0881b08c456df5a97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.505291 29599 leader_election.cc:290] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.505466 28902 rpcz_store.cc:276] 0320 15:27:37.221795 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.526075 (+3304280us) service_pool.cc:225] Handling call
0320 15:27:40.535915 (+ 9840us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:40.536067 (+ 152us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:40.536163 (+ 96us) catalog_manager.cc:2824] Locking tablets
0320 15:27:40.536349 (+ 186us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:40.564173 (+ 27824us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:40.564278 (+ 105us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:41.449893 (+885615us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:41.504014 (+ 54121us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.540398 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.540559 (+ 161us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.540568 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:40.540573 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:40.540715 (+ 142us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.540733 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.540738 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:40.540880 (+ 142us) tablet.cc:700] Row locks acquired
0320 15:27:40.540883 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:40.540933 (+ 50us) write_op.cc:270] Start()
0320 15:27:40.540972 (+ 39us) write_op.cc:276] Timestamp: P: 1774020460540879 usec, L: 0
0320 15:27:40.540976 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.541267 (+ 291us) log.cc:844] Serialized 725 byte log entry
0320 15:27:40.545646 (+ 4379us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.545781 (+ 135us) write_op.cc:301] APPLY: starting
0320 15:27:40.545797 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.545868 (+ 71us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.545906 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.545972 (+ 66us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.546011 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:40.546020 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:40.546110 (+ 90us) log.cc:844] Serialized 29 byte log entry
0320 15:27:40.546183 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.546210 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:40.546220 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.546334 (+ 114us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.546339 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":81,"apply.run_cpu_time_us":537,"apply.run_wall_time_us":536,"num_ops":2,"prepare.queue_time_us":3223,"prepare.run_cpu_time_us":1003,"prepare.run_wall_time_us":1002,"raft.queue_time_us":376,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":281,"replication_time_us":4632}]]}
I20260320 15:27:41.506431 28902 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_3c050d31cc0248c7beb57aafd767d53f" } modify_external_catalogs: true
W20260320 15:27:41.507990 28524 leader_election.cc:343] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:41.508225 28524 leader_election.cc:304] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:41.510753 29599 raft_consensus.cc:2749] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:41.511513 28524 consensus_peers.cc:597] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3bc4dd47817490d88725127b456cc70. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.512894 30153 raft_consensus.cc:515] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.515858 30153 leader_election.cc:290] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.494599 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8241a342054240d2ac62b729cc9c9a62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.516336 28443 raft_consensus.cc:3060] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.502074 28734 tablet_bootstrap.cc:654] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.506130 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 87cb35cefee841d0881b08c456df5a97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:41.508852 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.519340 28444 raft_consensus.cc:2468] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:41.519428 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.520293 28443 raft_consensus.cc:2468] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.520886 28240 leader_election.cc:304] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.521211 30256 raft_consensus.cc:2804] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.521338 30256 raft_consensus.cc:697] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:41.521634 30256 consensus_queue.cc:237] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.521934 29643 log.cc:1199] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/fc009401bb3d440f9d36e080efef1204
I20260320 15:27:41.510509 28445 raft_consensus.cc:2468] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:41.522300 29643 ts_tablet_manager.cc:1950] T fc009401bb3d440f9d36e080efef1204 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.522728 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 132) took 7319 ms. Trace:
I20260320 15:27:41.522923 29643 rpcz_store.cc:276] 0320 15:27:34.203737 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.204287 (+ 550us) service_pool.cc:225] Handling call
0320 15:27:41.484483 (+7280196us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:41.484493 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.489376 (+ 4883us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.492523 (+ 3147us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.522710 (+ 30187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7279984}
I20260320 15:27:41.524415 29643 tablet_replica.cc:333] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.524886 29599 raft_consensus.cc:493] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.525022 29599 raft_consensus.cc:515] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.525367 29643 raft_consensus.cc:2243] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.525533 29643 raft_consensus.cc:2272] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.525923 29599 leader_election.cc:290] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.526383 28524 leader_election.cc:304] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.526669 29599 raft_consensus.cc:2804] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.526773 29599 raft_consensus.cc:493] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.526865 29599 raft_consensus.cc:3060] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.527127 30257 raft_consensus.cc:493] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.527278 30257 raft_consensus.cc:515] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.527951 30257 leader_election.cc:290] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.530632 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.478255 29678 log.cc:1199] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7be03035dc3d44f9aa5b43b49281a940
I20260320 15:27:41.534590 29678 ts_tablet_manager.cc:1950] T 7be03035dc3d44f9aa5b43b49281a940 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:41.535949 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.537209 29599 raft_consensus.cc:515] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.538492 29599 leader_election.cc:290] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:41.539743 28524 leader_election.cc:343] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: STOPPED
I20260320 15:27:41.540619 28733 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.540776 28733 ts_tablet_manager.cc:1403] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.234s user 0.002s sys 0.016s
I20260320 15:27:41.541549 29643 ts_tablet_manager.cc:1916] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.541581 28733 raft_consensus.cc:359] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.541751 28733 raft_consensus.cc:385] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.541805 28733 raft_consensus.cc:740] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.542026 28733 consensus_queue.cc:260] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.542598 28733 ts_tablet_manager.cc:1434] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:41.543171 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 87cb35cefee841d0881b08c456df5a97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:41.544396 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.544903 28445 raft_consensus.cc:2468] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:41.547518 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.547649 28297 raft_consensus.cc:3060] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.547988 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc009401bb3d440f9d36e080efef1204" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:41.548131 28303 raft_consensus.cc:3060] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.550280 28734 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.550508 28734 ts_tablet_manager.cc:1403] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.003s
I20260320 15:27:41.550570 28303 raft_consensus.cc:2468] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:41.550801 30153 raft_consensus.cc:493] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.550983 30153 raft_consensus.cc:515] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.551692 30153 leader_election.cc:290] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.552541 30170 consensus_queue.cc:1048] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.552736 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.552793 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 132) took 7392 ms. Trace:
I20260320 15:27:41.552913 28443 raft_consensus.cc:3060] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.552875 29678 rpcz_store.cc:276] 0320 15:27:34.160816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160902 (+ 86us) service_pool.cc:225] Handling call
0320 15:27:41.325214 (+7164312us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:41.325228 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.406213 (+ 80985us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.467849 (+ 61636us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.552775 (+ 84926us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5888,"tablet-delete.queue_time_us":7147997}
I20260320 15:27:41.553290 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 134) took 7391 ms. Trace:
I20260320 15:27:41.553372 29678 rpcz_store.cc:276] 0320 15:27:34.161949 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.162128 (+ 179us) service_pool.cc:225] Handling call
0320 15:27:41.553156 (+7391028us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:41.553166 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.553264 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7375788}
W20260320 15:27:41.553514 28524 leader_election.cc:343] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.553676 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 128) took 7394 ms. Trace:
I20260320 15:27:41.553754 29678 rpcz_store.cc:276] 0320 15:27:34.159612 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.159781 (+ 169us) service_pool.cc:225] Handling call
0320 15:27:41.553571 (+7393790us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:41.553579 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.553660 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7375994}
I20260320 15:27:41.554049 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 126) took 7394 ms. Trace:
I20260320 15:27:41.554020 28524 leader_election.cc:304] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:41.554116 29678 rpcz_store.cc:276] 0320 15:27:34.159283 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.160277 (+ 994us) service_pool.cc:225] Handling call
0320 15:27:41.553940 (+7393663us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:41.553950 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.554030 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7375962}
I20260320 15:27:41.554358 29678 tablet_replica.cc:333] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:41.554576 29678 raft_consensus.cc:2243] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.554906 29678 raft_consensus.cc:2272] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.554826 28734 raft_consensus.cc:359] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.555222 28734 raft_consensus.cc:385] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.555408 28734 raft_consensus.cc:740] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.555702 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.555899 28440 raft_consensus.cc:2468] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:41.556444 28443 raft_consensus.cc:2468] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:41.557060 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.557228 28736 tablet_bootstrap.cc:492] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.557442 28736 ts_tablet_manager.cc:1403] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.003s
I20260320 15:27:41.557691 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:41.558296 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.558888 28736 raft_consensus.cc:359] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.559154 28736 raft_consensus.cc:385] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.559281 28736 raft_consensus.cc:740] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.559592 28736 consensus_queue.cc:260] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.560509 28736 ts_tablet_manager.cc:1434] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:41.559715 28734 consensus_queue.cc:260] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.563978 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771bd6a3f0e47d99a0191eae25cb2d2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.564255 28576 raft_consensus.cc:2393] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:41.564666 28524 leader_election.cc:304] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.565132 28239 leader_election.cc:304] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:41.565464 28524 leader_election.cc:304] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.565866 29939 raft_consensus.cc:2804] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.565976 29939 raft_consensus.cc:697] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:41.566212 29939 consensus_queue.cc:237] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.566594 30153 raft_consensus.cc:2804] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.566694 30153 raft_consensus.cc:493] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.566762 30153 raft_consensus.cc:3060] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.567222 30155 raft_consensus.cc:2804] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.567335 30155 raft_consensus.cc:493] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.567407 30155 raft_consensus.cc:3060] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.569700 30153 raft_consensus.cc:515] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.570072 30155 raft_consensus.cc:515] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.570125 28734 ts_tablet_manager.cc:1434] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.019s user 0.000s sys 0.003s
I20260320 15:27:41.570478 30153 leader_election.cc:290] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.570603 30155 leader_election.cc:290] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.571465 28733 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.573285 28733 tablet_bootstrap.cc:654] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.572484 28734 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.579027 29643 ts_tablet_manager.cc:1929] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:41.585016 28902 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_3c050d31cc0248c7beb57aafd767d53f" } modify_external_catalogs: true
I20260320 15:27:41.588406 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 88) took 5729 ms. Trace:
I20260320 15:27:41.588740 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b2355eadcae6483bae7ae475d74dd131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9176e734212849aab8d99473de02cc79) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.584883 30257 raft_consensus.cc:2749] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.584844 29678 ts_tablet_manager.cc:1916] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.593248 30170 consensus_queue.cc:1048] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.596092 29643 log.cc:1199] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:41.596529 29643 ts_tablet_manager.cc:1950] T 8241a342054240d2ac62b729cc9c9a62 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.589020 28906 rpcz_store.cc:276] 0320 15:27:35.804209 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.410771 (+4606562us) service_pool.cc:225] Handling call
0320 15:27:40.432727 (+ 21956us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:40.432972 (+ 245us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:40.455871 (+ 22899us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:40.456013 (+ 142us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:40.499214 (+ 43201us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:41.533597 (+1034383us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.488891 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.489022 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.489029 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:40.489031 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:40.489128 (+ 97us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.489140 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.489145 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:40.489222 (+ 77us) tablet.cc:700] Row locks acquired
0320 15:27:40.489224 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:40.489266 (+ 42us) write_op.cc:270] Start()
0320 15:27:40.489306 (+ 40us) write_op.cc:276] Timestamp: P: 1774020460489218 usec, L: 0
0320 15:27:40.489310 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.489525 (+ 215us) log.cc:844] Serialized 405 byte log entry
0320 15:27:40.496422 (+ 6897us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.496769 (+ 347us) write_op.cc:301] APPLY: starting
0320 15:27:40.496789 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.496834 (+ 45us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.496860 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.498726 (+ 1866us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.498766 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:40.498905 (+ 139us) write_op.cc:312] APPLY: finished
0320 15:27:40.499007 (+ 102us) log.cc:844] Serialized 23 byte log entry
0320 15:27:40.499084 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.499107 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:40.499115 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.499209 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.499213 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14054,"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":670,"apply.run_wall_time_us":2431,"num_ops":1,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":705,"prepare.run_wall_time_us":705,"replication_time_us":7079,"spinlock_wait_cycles":9600}]]}
I20260320 15:27:41.597698 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_5b5bbe0346774e8cbb24c184595d322e" } new_table_name: "default.Table_96e808cfcc8442e2bce7aa1f13986b23" modify_external_catalogs: true
I20260320 15:27:41.608919 29640 ts_tablet_manager.cc:1929] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.609061 29640 log.cc:1199] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:41.609449 29640 ts_tablet_manager.cc:1950] T a2775196c8e04dcc80c7e089b57579fd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.610268 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 121) took 7609 ms. Trace:
I20260320 15:27:41.610376 29640 rpcz_store.cc:276] 0320 15:27:34.000873 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.000936 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:41.421327 (+7420391us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:41.421649 (+ 322us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.545948 (+124299us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.608790 (+ 62842us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.610237 (+ 1447us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7419593}
I20260320 15:27:41.616954 28733 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.617120 28733 ts_tablet_manager.cc:1403] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.000s
I20260320 15:27:41.617841 28733 raft_consensus.cc:359] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.618016 28733 raft_consensus.cc:385] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.618062 28733 raft_consensus.cc:740] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.618270 28733 consensus_queue.cc:260] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.618749 28733 ts_tablet_manager.cc:1434] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:41.619491 28733 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:41.621269 28903 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e779a274913f4caeaf82e3f180457830 consensus state has no leader
I20260320 15:27:41.597343 28734 tablet_bootstrap.cc:654] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.629137 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 134) took 7425 ms. Trace:
I20260320 15:27:41.629532 29643 rpcz_store.cc:276] 0320 15:27:34.203942 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.206194 (+ 2252us) service_pool.cc:225] Handling call
0320 15:27:41.524346 (+7318152us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:41.524361 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.574424 (+ 50063us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.578878 (+ 4454us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.629065 (+ 50187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7317859}
I20260320 15:27:41.528190 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet fc009401bb3d440f9d36e080efef1204 (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]) successfully deleted
W20260320 15:27:41.635800 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 70367.2us
I20260320 15:27:41.636766 28524 leader_election.cc:304] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.639029 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.639225 28445 raft_consensus.cc:3060] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.641911 28445 raft_consensus.cc:2468] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:41.643589 29939 raft_consensus.cc:2804] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.643899 29939 raft_consensus.cc:697] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:41.644145 29643 tablet_replica.cc:333] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.644255 29939 consensus_queue.cc:237] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.644562 28734 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.644641 29643 raft_consensus.cc:2243] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.644898 28734 ts_tablet_manager.cc:1403] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.009s
I20260320 15:27:41.645229 30170 raft_consensus.cc:493] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:41.645364 30170 raft_consensus.cc:515] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.646060 30170 leader_election.cc:290] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:41.647686 28382 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
W20260320 15:27:41.647945 28383 leader_election.cc:343] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:41.648036 28383 leader_election.cc:304] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.648571 30170 raft_consensus.cc:2749] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260320 15:27:41.650092 28734 raft_consensus.cc:359] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.650535 28734 raft_consensus.cc:385] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.650734 28734 raft_consensus.cc:740] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.651263 28734 consensus_queue.cc:260] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.652660 28734 ts_tablet_manager.cc:1434] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
W20260320 15:27:41.653612 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 107524us
I20260320 15:27:41.654803 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7be03035dc3d44f9aa5b43b49281a940 (table default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]) successfully deleted
W20260320 15:27:41.655280 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41da3cb06b1147fbae2695bc5c9ffaa5 already in progress: creating tablet
W20260320 15:27:41.656141 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6d9b0f1936f54c699211c5ee9709f9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d9b0f1936f54c699211c5ee9709f9ed already in progress: creating tablet
W20260320 15:27:41.656796 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3ccbf4f32636463f8e1e005a629b92cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ccbf4f32636463f8e1e005a629b92cc already in progress: creating tablet
I20260320 15:27:41.657500 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8241a342054240d2ac62b729cc9c9a62 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
I20260320 15:27:41.611029 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet a2775196c8e04dcc80c7e089b57579fd (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]) successfully deleted
I20260320 15:27:41.658797 28733 tablet_bootstrap.cc:654] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.658811 28734 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:41.658862 28872 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 47952.1us
I20260320 15:27:41.644958 29643 raft_consensus.cc:2272] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.662042 29643 ts_tablet_manager.cc:1916] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.662326 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.663605 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 26818e620ff649d18b671e4f4c7c7e74 with delete_type TABLET_DATA_DELETED (replaced by tablet 8241a342054240d2ac62b729cc9c9a62) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.664522 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet a2775196c8e04dcc80c7e089b57579fd with delete_type TABLET_DATA_DELETED (replaced by tablet 8eb3d5e19cda48fda124b6238902c03a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.665244 29640 tablet_replica.cc:333] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.665298 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.665467 29640 raft_consensus.cc:2243] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.665659 29640 raft_consensus.cc:2272] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.666257 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a072859f39c4c28ae65b0e9afa79d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:41.666440 29640 ts_tablet_manager.cc:1916] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:41.677563 28903 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 95) took 3643 ms (client timeout 1000 ms). Trace:
I20260320 15:27:41.677759 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet ea95b7a1b30e42cebcbfcd551c58f679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:41.677747 28903 rpcz_store.cc:269] 0320 15:27:38.033131 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.991990 (+958859us) service_pool.cc:225] Handling call
0320 15:27:41.676906 (+2684916us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.491694 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.491844 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.491852 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:39.491855 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:39.491996 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.492010 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.492015 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.492092 (+ 77us) tablet.cc:700] Row locks acquired
0320 15:27:39.492096 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:39.492142 (+ 46us) write_op.cc:270] Start()
0320 15:27:39.492179 (+ 37us) write_op.cc:276] Timestamp: P: 1774020459492085 usec, L: 0
0320 15:27:39.492183 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.492427 (+ 244us) log.cc:844] Serialized 527 byte log entry
0320 15:27:39.499769 (+ 7342us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.500511 (+ 742us) write_op.cc:301] APPLY: starting
0320 15:27:39.500532 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.500626 (+ 94us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.500659 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.500721 (+ 62us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.500757 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.500764 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:39.500846 (+ 82us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.500914 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.500935 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:39.500943 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.501026 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.501029 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:39.536575 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.536724 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.536732 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:39.536735 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:39.536849 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.536866 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.536871 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:39.536936 (+ 65us) tablet.cc:700] Row locks acquired
0320 15:27:39.536939 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:39.536978 (+ 39us) write_op.cc:270] Start()
0320 15:27:39.537008 (+ 30us) write_op.cc:276] Timestamp: P: 1774020459536920 usec, L: 0
0320 15:27:39.537012 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.537228 (+ 216us) log.cc:844] Serialized 356 byte log entry
0320 15:27:39.540802 (+ 3574us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.541060 (+ 258us) write_op.cc:301] APPLY: starting
0320 15:27:39.541079 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.541125 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.541162 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.541219 (+ 57us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.541254 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:39.541261 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:39.541346 (+ 85us) log.cc:844] Serialized 23 byte log entry
0320 15:27:39.541413 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.541435 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:39.541441 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.541520 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.541524 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":304893,"reported_tablets":213,"child_traces":[["op",{"apply.queue_time_us":670,"apply.run_cpu_time_us":520,"apply.run_wall_time_us":520,"num_ops":1,"prepare.queue_time_us":11950,"prepare.run_cpu_time_us":831,"prepare.run_wall_time_us":830,"replication_time_us":7554}],["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":445,"apply.run_wall_time_us":444,"num_ops":1,"prepare.queue_time_us":15359,"prepare.run_cpu_time_us":792,"prepare.run_wall_time_us":793,"raft.queue_time_us":334,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":265,"replication_time_us":3758}]]}
I20260320 15:27:41.679009 28734 tablet_bootstrap.cc:654] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.680598 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_c66e0ce947a146eeafcac51f268deaa1" } new_table_name: "default.Table_283c1ab1e64449abaa1696a2d2bc7911" modify_external_catalogs: true
W20260320 15:27:41.692116 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 96) took 2459 ms (client timeout 1000 ms). Trace:
W20260320 15:27:41.692378 28904 rpcz_store.cc:269] 0320 15:27:39.233060 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.272639 (+ 39579us) service_pool.cc:225] Handling call
0320 15:27:41.692053 (+2419414us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:39.990902 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:39.991049 (+ 147us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:39.991059 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:39.991064 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:39.991303 (+ 239us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:39.991327 (+ 24us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:39.991337 (+ 10us) tablet.cc:684] Acquiring locks for 3 operations
0320 15:27:39.991512 (+ 175us) tablet.cc:700] Row locks acquired
0320 15:27:39.991517 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:39.991606 (+ 89us) write_op.cc:270] Start()
0320 15:27:39.991651 (+ 45us) write_op.cc:276] Timestamp: P: 1774020459991548 usec, L: 0
0320 15:27:39.991656 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:39.991962 (+ 306us) log.cc:844] Serialized 1275 byte log entry
0320 15:27:39.997223 (+ 5261us) op_driver.cc:464] REPLICATION: finished
0320 15:27:39.997382 (+ 159us) write_op.cc:301] APPLY: starting
0320 15:27:39.997404 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:39.997474 (+ 70us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:39.997565 (+ 91us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:39.997672 (+ 107us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:39.997722 (+ 50us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0320 15:27:39.997730 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:39.997836 (+ 106us) log.cc:844] Serialized 35 byte log entry
0320 15:27:39.997912 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.997939 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:39.997948 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:39.998045 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:39.998049 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27792,"reported_tablets":200,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":650,"apply.run_wall_time_us":649,"num_ops":3,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":1202,"prepare.run_wall_time_us":1211,"raft.queue_time_us":614,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":449,"replication_time_us":5487}]]}
I20260320 15:27:41.694168 28733 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.694326 28733 ts_tablet_manager.cc:1403] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.001s
I20260320 15:27:41.696928 28733 raft_consensus.cc:359] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.697135 28733 raft_consensus.cc:385] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.697203 28733 raft_consensus.cc:740] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.697443 28733 consensus_queue.cc:260] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.698014 28733 ts_tablet_manager.cc:1434] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260320 15:27:41.698326 28733 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.698586 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_263dcedacaae47cca5681f61731b53d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:41.701205 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_263dcedacaae47cca5681f61731b53d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:41.637917 28736 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:41.706875 30289 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.709043 30290 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.708719 29643 ts_tablet_manager.cc:1929] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:41.709661 29643 log.cc:1199] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/56fa691a26114657ad768545a30dfd3c
I20260320 15:27:41.709690 29678 ts_tablet_manager.cc:1929] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.711440 29678 log.cc:1199] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9176e734212849aab8d99473de02cc79
I20260320 15:27:41.712877 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:41.713152 28443 raft_consensus.cc:3060] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.713263 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4b5a658cd84b04b50b54973c065889" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:41.713447 28303 raft_consensus.cc:3060] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.714732 29842 raft_consensus.cc:493] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.714857 29842 raft_consensus.cc:515] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.715517 29842 leader_election.cc:290] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.715982 28443 raft_consensus.cc:2468] T 01078353a3a343a3a1231245a391cf8e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:41.716281 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:41.716821 28382 leader_election.cc:343] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.716075 29678 ts_tablet_manager.cc:1950] T 9176e734212849aab8d99473de02cc79 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:41.716913 28733 tablet_bootstrap.cc:654] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.718008 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01078353a3a343a3a1231245a391cf8e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:41.720588 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 138) took 7551 ms. Trace:
I20260320 15:27:41.721908 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:41.722442 28383 leader_election.cc:343] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.722585 28383 leader_election.cc:304] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.723021 29842 raft_consensus.cc:2749] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.723608 30287 tablet.cc:1723] T 7a8e822249154a1784ebd66f8e56abdf P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.736100 28303 raft_consensus.cc:2468] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:41.737367 29599 raft_consensus.cc:493] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.737538 29599 raft_consensus.cc:515] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.738143 29599 leader_election.cc:290] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.749876 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:41.721449 29678 rpcz_store.cc:276] 0320 15:27:34.169698 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.172167 (+ 2469us) service_pool.cc:225] Handling call
0320 15:27:41.554315 (+7382148us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:41.554324 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.651625 (+ 97301us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.709042 (+ 57417us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.719748 (+ 10706us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7371798}
I20260320 15:27:41.751653 29678 tablet_replica.cc:333] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:41.751660 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9176e734212849aab8d99473de02cc79 (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:41.751869 29678 raft_consensus.cc:2243] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.752034 29678 raft_consensus.cc:2272] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.727931 29643 ts_tablet_manager.cc:1950] T 56fa691a26114657ad768545a30dfd3c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.754056 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 135) took 7534 ms. Trace:
I20260320 15:27:41.754186 29643 rpcz_store.cc:276] 0320 15:27:34.219919 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.220117 (+ 198us) service_pool.cc:225] Handling call
0320 15:27:41.630916 (+7410799us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:41.644070 (+ 13154us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.682432 (+ 38362us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.707932 (+ 25500us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.754026 (+ 46094us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7410456}
I20260320 15:27:41.754632 29643 tablet_replica.cc:333] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.754855 29643 raft_consensus.cc:2243] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.755014 29643 raft_consensus.cc:2272] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.755682 29643 ts_tablet_manager.cc:1916] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.758108 29599 consensus_queue.cc:1048] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:41.758913 28524 leader_election.cc:343] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:41.759274 28524 leader_election.cc:343] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.729666 28734 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.759377 29678 ts_tablet_manager.cc:1916] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.759686 28734 ts_tablet_manager.cc:1403] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.107s user 0.004s sys 0.007s
I20260320 15:27:41.759913 28736 tablet_bootstrap.cc:654] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.760568 28734 raft_consensus.cc:359] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.760751 28734 raft_consensus.cc:385] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.760805 28734 raft_consensus.cc:740] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.761014 28734 consensus_queue.cc:260] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.761584 28734 ts_tablet_manager.cc:1434] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:41.761896 28734 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.763901 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 56fa691a26114657ad768545a30dfd3c (table default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]) successfully deleted
I20260320 15:27:41.765816 28734 tablet_bootstrap.cc:654] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.771785 29640 ts_tablet_manager.cc:1929] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.772125 29640 log.cc:1199] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b6201b74fdb0428e98f2eebb0b24d01a
I20260320 15:27:41.772974 29640 ts_tablet_manager.cc:1950] T b6201b74fdb0428e98f2eebb0b24d01a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.775120 28733 tablet_bootstrap.cc:492] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.775281 28733 ts_tablet_manager.cc:1403] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.002s
I20260320 15:27:41.776059 28733 raft_consensus.cc:359] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.776250 28733 raft_consensus.cc:385] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.776314 28733 raft_consensus.cc:740] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.776588 28733 consensus_queue.cc:260] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.777156 28733 ts_tablet_manager.cc:1434] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:41.777490 28733 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.800740 29643 ts_tablet_manager.cc:1929] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:41.802122 29643 log.cc:1199] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/07af28f257d745a1bb4173aad09045d9
W20260320 15:27:41.759522 28524 leader_election.cc:343] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:41.804952 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 45428.1us
W20260320 15:27:41.805318 28524 leader_election.cc:343] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.805727 28524 leader_election.cc:304] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:41.806586 29599 raft_consensus.cc:2804] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.806778 29599 raft_consensus.cc:697] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:41.807116 29599 consensus_queue.cc:237] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.816272 29939 raft_consensus.cc:493] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.816679 29939 raft_consensus.cc:515] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.818696 29939 leader_election.cc:290] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.820302 28734 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.820472 28734 ts_tablet_manager.cc:1403] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.059s user 0.005s sys 0.002s
I20260320 15:27:41.821384 28734 raft_consensus.cc:359] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.821595 28734 raft_consensus.cc:385] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.821655 28734 raft_consensus.cc:740] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.823783 28733 tablet_bootstrap.cc:654] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.822005 28734 consensus_queue.cc:260] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.830998 29643 ts_tablet_manager.cc:1950] T 07af28f257d745a1bb4173aad09045d9 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.831864 28734 ts_tablet_manager.cc:1434] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.002s sys 0.001s
I20260320 15:27:41.832206 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 124) took 7820 ms. Trace:
I20260320 15:27:41.836788 28734 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.834018 29640 rpcz_store.cc:276] 0320 15:27:34.011393 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.011446 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:41.665179 (+7653733us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:41.665194 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.717911 (+ 52717us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.771481 (+ 53570us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.831891 (+ 60410us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7653134}
I20260320 15:27:41.836529 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 136) took 7616 ms. Trace:
I20260320 15:27:41.838243 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b6201b74fdb0428e98f2eebb0b24d01a (table default.Table_99a1f24aff574bff820f96a585a8c417 [id=9d751447357540d3bae34b1e0ae5ceef]) successfully deleted
I20260320 15:27:41.839591 29640 tablet_replica.cc:333] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.837989 29643 rpcz_store.cc:276] 0320 15:27:34.220064 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.220475 (+ 411us) service_pool.cc:225] Handling call
0320 15:27:41.754581 (+7534106us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:41.754593 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.759348 (+ 4755us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.788499 (+ 29151us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.836195 (+ 47696us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7533896}
I20260320 15:27:41.842001 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 07af28f257d745a1bb4173aad09045d9 (table default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]) successfully deleted
I20260320 15:27:41.841647 29640 raft_consensus.cc:2243] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:41.846570 29640 raft_consensus.cc:2272] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.847705 29643 tablet_replica.cc:333] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.847936 30290 tablet.cc:1723] T 80810790e42947a69ce6140feb9f7cbc P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.849035 30289 tablet.cc:1723] T 80810790e42947a69ce6140feb9f7cbc P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.861596 28734 tablet_bootstrap.cc:654] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.863343 29640 ts_tablet_manager.cc:1916] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.866377 30170 raft_consensus.cc:493] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.864635 30302 tablet.cc:1723] T 80810790e42947a69ce6140feb9f7cbc P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:41.866680 30170 raft_consensus.cc:515] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:41.867846 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd315b5d49104d679b03515b65ee30ac" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.868521 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd315b5d49104d679b03515b65ee30ac" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:41.868362 28383 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:41.869704 28382 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:41.869823 28382 leader_election.cc:304] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:41.870157 29842 raft_consensus.cc:2749] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:41.868212 30170 leader_election.cc:290] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:41.872501 28733 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.872664 28733 ts_tablet_manager.cc:1403] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.095s user 0.005s sys 0.000s
I20260320 15:27:41.873371 28733 raft_consensus.cc:359] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.873555 28733 raft_consensus.cc:385] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.873608 28733 raft_consensus.cc:740] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.873813 28733 consensus_queue.cc:260] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.875308 28733 ts_tablet_manager.cc:1434] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:41.876223 28733 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.884524 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.884749 28443 raft_consensus.cc:2468] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:41.884915 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:41.885150 28440 raft_consensus.cc:2393] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:41.888623 28734 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.889067 28734 ts_tablet_manager.cc:1403] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.003s
I20260320 15:27:41.890251 28734 raft_consensus.cc:359] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.890677 28734 raft_consensus.cc:385] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.890868 28734 raft_consensus.cc:740] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.891393 28734 consensus_queue.cc:260] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.893878 28734 ts_tablet_manager.cc:1434] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:41.896461 28733 tablet_bootstrap.cc:654] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.897032 28734 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:41.885527 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54266dd376164673bffe829dd515b83d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.883523 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:41.877789 29678 ts_tablet_manager.cc:1929] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:41.902971 29678 log.cc:1199] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/fc009401bb3d440f9d36e080efef1204
I20260320 15:27:41.903364 29678 ts_tablet_manager.cc:1950] T fc009401bb3d440f9d36e080efef1204 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:41.903789 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 137) took 7735 ms. Trace:
I20260320 15:27:41.903888 29678 rpcz_store.cc:276] 0320 15:27:34.168406 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.168459 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:41.751594 (+7583135us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:41.751607 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.834595 (+ 82988us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.877666 (+ 43071us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.903771 (+ 26105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7567662}
I20260320 15:27:41.904284 29678 tablet_replica.cc:333] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:41.904479 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet fc009401bb3d440f9d36e080efef1204 (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]) successfully deleted
I20260320 15:27:41.876405 28736 tablet_bootstrap.cc:492] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:41.904945 28736 ts_tablet_manager.cc:1403] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.268s user 0.005s sys 0.001s
I20260320 15:27:41.905915 28736 raft_consensus.cc:359] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.906137 28736 raft_consensus.cc:385] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.906304 28736 raft_consensus.cc:740] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.906785 28736 consensus_queue.cc:260] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.909996 28734 tablet_bootstrap.cc:654] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:41.918278 29643 raft_consensus.cc:2243] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.918553 29643 raft_consensus.cc:2272] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.919219 29643 ts_tablet_manager.cc:1916] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.919521 28524 leader_election.cc:304] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:41.921772 30153 raft_consensus.cc:2804] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:41.921942 30153 raft_consensus.cc:697] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:41.922305 30153 consensus_queue.cc:237] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.927845 29842 consensus_queue.cc:1048] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.929606 28733 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:41.930061 28733 ts_tablet_manager.cc:1403] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.000s
I20260320 15:27:41.933483 28733 raft_consensus.cc:359] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.934839 28733 raft_consensus.cc:385] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.936606 28733 raft_consensus.cc:740] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.939090 28733 consensus_queue.cc:260] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.942957 29643 ts_tablet_manager.cc:1929] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.943305 29643 log.cc:1199] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ecaa08ba34514c5e99a9e2d4ba9268a9
I20260320 15:27:41.944603 29643 ts_tablet_manager.cc:1950] T ecaa08ba34514c5e99a9e2d4ba9268a9 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:41.947891 28733 ts_tablet_manager.cc:1434] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260320 15:27:41.952307 28734 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:41.952685 28734 ts_tablet_manager.cc:1403] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.056s user 0.010s sys 0.002s
I20260320 15:27:41.952683 28733 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:41.954771 28734 raft_consensus.cc:359] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.955247 28734 raft_consensus.cc:385] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:41.954761 29640 ts_tablet_manager.cc:1929] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:41.955458 28734 raft_consensus.cc:740] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:41.956146 28734 consensus_queue.cc:260] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.957448 28734 ts_tablet_manager.cc:1434] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:41.958316 30153 raft_consensus.cc:493] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.958472 30153 raft_consensus.cc:515] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.959080 30153 leader_election.cc:290] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:41.959859 28904 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 99) took 2323 ms. Trace:
I20260320 15:27:41.960053 28904 rpcz_store.cc:276] 0320 15:27:39.636023 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.695273 (+2059250us) service_pool.cc:225] Handling call
0320 15:27:41.745834 (+ 50561us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:41.746499 (+ 665us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:41.813270 (+ 66771us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:41.959630 (+146360us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:41.959775 (+ 145us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:41.783693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:41.783865 (+ 172us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:41.783874 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:41.783877 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:41.783994 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:41.784009 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:41.784016 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:41.784116 (+ 100us) tablet.cc:700] Row locks acquired
0320 15:27:41.784119 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:41.784186 (+ 67us) write_op.cc:270] Start()
0320 15:27:41.784232 (+ 46us) write_op.cc:276] Timestamp: P: 1774020461784140 usec, L: 0
0320 15:27:41.784236 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:41.785922 (+ 1686us) log.cc:844] Serialized 433 byte log entry
0320 15:27:41.786319 (+ 397us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:41.793936 (+ 7617us) op_driver.cc:464] REPLICATION: finished
0320 15:27:41.795760 (+ 1824us) write_op.cc:301] APPLY: starting
0320 15:27:41.795776 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:41.795818 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:41.795822 (+ 4us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:41.795958 (+ 136us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:41.795990 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:41.795997 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:41.796074 (+ 77us) log.cc:844] Serialized 29 byte log entry
0320 15:27:41.812959 (+ 16885us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.812999 (+ 40us) write_op.cc:454] Released schema lock
0320 15:27:41.813014 (+ 15us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:41.813131 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.813135 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1768,"apply.run_cpu_time_us":616,"apply.run_wall_time_us":17354,"num_ops":2,"prepare.queue_time_us":36439,"prepare.run_cpu_time_us":1390,"prepare.run_wall_time_us":2723,"raft.queue_time_us":36,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":308,"replication_time_us":9667,"spinlock_wait_cycles":3413376,"thread_start_us":1211,"threads_started":1}]]}
W20260320 15:27:41.963197 28524 consensus_peers.cc:597] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fa5f6fe801b4452bf294a9ae23bb163. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:41.964063 30170 raft_consensus.cc:493] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:41.964236 30170 raft_consensus.cc:515] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:41.964514 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 137) took 7743 ms. Trace:
I20260320 15:27:41.964608 29643 rpcz_store.cc:276] 0320 15:27:34.220853 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.220898 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:41.847647 (+7626749us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:41.847657 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.931082 (+ 83425us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.942717 (+ 11635us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.964489 (+ 21772us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70167,"tablet-delete.queue_time_us":7626521}
I20260320 15:27:41.965212 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (table default.Table_30015eb07aba409bb03a37140eea20d6 [id=83739ddc977f4089b85ad8e6ea64df8b]) successfully deleted
I20260320 15:27:41.967144 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:41.968384 29939 consensus_queue.cc:1048] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.969436 29640 log.cc:1199] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/583ed053b2a742c7b029f348f4495101
I20260320 15:27:41.970808 29939 consensus_queue.cc:1048] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:41.971508 28524 leader_election.cc:304] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:41.971837 29643 tablet_replica.cc:333] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:41.972054 29643 raft_consensus.cc:2243] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.972222 29643 raft_consensus.cc:2272] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.972803 29643 ts_tablet_manager.cc:1916] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:41.974081 29640 ts_tablet_manager.cc:1950] T 583ed053b2a742c7b029f348f4495101 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:41.976058 29939 raft_consensus.cc:2804] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:41.976159 29939 raft_consensus.cc:493] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:41.976225 29939 raft_consensus.cc:3060] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:41.977949 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 126) took 7965 ms. Trace:
I20260320 15:27:41.980180 29640 rpcz_store.cc:276] 0320 15:27:34.012489 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.012539 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:41.839117 (+7826578us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:41.839135 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.908699 (+ 69564us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.953695 (+ 44996us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:41.977918 (+ 24223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7825885}
I20260320 15:27:41.983737 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 583ed053b2a742c7b029f348f4495101 (table default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]) successfully deleted
I20260320 15:27:41.983969 29678 raft_consensus.cc:2243] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:41.984416 29678 pending_rounds.cc:70] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:41.984464 29678 pending_rounds.cc:77] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387806270791680 op_type: NO_OP noop_request { }
I20260320 15:27:41.984582 29678 raft_consensus.cc:2889] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:41.984706 29678 raft_consensus.cc:2272] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:41.985407 29640 tablet_replica.cc:333] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:41.988914 29678 ts_tablet_manager.cc:1916] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:41.991372 28382 leader_election.cc:343] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.003108 29678 ts_tablet_manager.cc:1929] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.003443 29678 log.cc:1199] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/bdd8f5ccbd35418684fd27459c7a6b9f
I20260320 15:27:42.003504 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.003609 30170 leader_election.cc:290] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.004112 30168 raft_consensus.cc:493] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.026166 30168 raft_consensus.cc:515] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.027015 30168 leader_election.cc:290] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:42.035677 28239 consensus_peers.cc:597] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.038450 29643 ts_tablet_manager.cc:1929] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.041430 29643 log.cc:1199] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/07ff84dedc074dabb4e8e2d8460066e7
I20260320 15:27:42.042085 29643 ts_tablet_manager.cc:1950] T 07ff84dedc074dabb4e8e2d8460066e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.042477 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 138) took 7821 ms. Trace:
I20260320 15:27:42.042613 29643 rpcz_store.cc:276] 0320 15:27:34.221307 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.221359 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:41.964934 (+7743575us) ts_tablet_manager.cc:1151] Deleting tablet 07ff84dedc074dabb4e8e2d8460066e7
0320 15:27:41.971766 (+ 6832us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.009826 (+ 38060us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.038321 (+ 28495us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.042461 (+ 4140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7743372}
I20260320 15:27:42.008272 29939 raft_consensus.cc:515] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.010344 29640 raft_consensus.cc:2243] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.043016 29640 raft_consensus.cc:2272] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.043458 29939 leader_election.cc:290] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.043498 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]) successfully deleted
I20260320 15:27:42.043680 29640 ts_tablet_manager.cc:1916] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.043944 29643 tablet_replica.cc:333] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.044373 29643 raft_consensus.cc:2243] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:42.013767 28733 tablet_bootstrap.cc:654] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.029263 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.049495 30170 raft_consensus.cc:493] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.049664 30170 raft_consensus.cc:515] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.050279 30170 leader_election.cc:290] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.045194 28734 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.052238 28734 tablet_bootstrap.cc:654] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.052639 29643 raft_consensus.cc:2272] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.056627 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.057354 29643 ts_tablet_manager.cc:1916] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.060866 28584 raft_consensus.cc:3060] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:42.058914 28382 leader_election.cc:343] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:42.062162 28382 leader_election.cc:343] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.075191 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:42.076498 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:42.077844 28734 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.078006 28734 ts_tablet_manager.cc:1403] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20260320 15:27:42.078791 28734 raft_consensus.cc:359] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.079063 28734 raft_consensus.cc:385] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.079162 28734 raft_consensus.cc:740] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.079281 30170 raft_consensus.cc:493] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.079492 30170 raft_consensus.cc:515] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.080433 30170 leader_election.cc:290] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.079485 28734 consensus_queue.cc:260] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.081094 28734 ts_tablet_manager.cc:1434] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:42.081151 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.081513 28734 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:42.081846 28382 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.087116 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "889d620c311a42039c22af53ad0802f2" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.087316 30256 consensus_queue.cc:1048] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.092067 30170 raft_consensus.cc:493] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.094444 30170 raft_consensus.cc:515] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:42.094864 28890 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 97) took 1801 ms (client timeout 1000 ms). Trace:
I20260320 15:27:42.095012 28734 tablet_bootstrap.cc:654] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.096158 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.096910 28382 leader_election.cc:343] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.098012 30170 leader_election.cc:290] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:42.100111 28890 rpcz_store.cc:269] 0320 15:27:40.293587 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.343479 (+ 49892us) service_pool.cc:225] Handling call
0320 15:27:42.094785 (+1751306us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.880224 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.880374 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.880384 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:40.880388 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:40.880517 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.880536 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.880541 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:40.880615 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:40.880617 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:40.880664 (+ 47us) write_op.cc:270] Start()
0320 15:27:40.880705 (+ 41us) write_op.cc:276] Timestamp: P: 1774020460880623 usec, L: 0
0320 15:27:40.880709 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.880954 (+ 245us) log.cc:844] Serialized 527 byte log entry
0320 15:27:40.884906 (+ 3952us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.885029 (+ 123us) write_op.cc:301] APPLY: starting
0320 15:27:40.885049 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.885095 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.885136 (+ 41us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.885198 (+ 62us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.885233 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:40.885240 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:40.885328 (+ 88us) log.cc:844] Serialized 23 byte log entry
0320 15:27:40.885397 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.885421 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:40.885428 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.885521 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.885525 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":91985,"reported_tablets":194,"child_traces":[["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":470,"apply.run_wall_time_us":469,"num_ops":1,"prepare.queue_time_us":4731,"prepare.run_cpu_time_us":880,"prepare.run_wall_time_us":881,"raft.queue_time_us":572,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":248,"replication_time_us":4162}]]}
I20260320 15:27:42.102288 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_799abb7057d3405db01b42c86431f01d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.103960 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_799abb7057d3405db01b42c86431f01d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.106163 28734 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.109375 28734 ts_tablet_manager.cc:1403] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260320 15:27:42.115983 28734 raft_consensus.cc:359] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.118587 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 100) took 2479 ms. Trace:
I20260320 15:27:42.118935 29640 ts_tablet_manager.cc:1929] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.119086 29640 log.cc:1199] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:42.121551 28734 raft_consensus.cc:385] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.121718 28734 raft_consensus.cc:740] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.122077 28734 consensus_queue.cc:260] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.122675 28734 ts_tablet_manager.cc:1434] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260320 15:27:42.122759 28524 leader_election.cc:304] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:42.123301 28524 leader_election.cc:343] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.124446 29678 ts_tablet_manager.cc:1950] T bdd8f5ccbd35418684fd27459c7a6b9f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.124670 29939 raft_consensus.cc:2749] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.125574 28734 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.126698 28890 rpcz_store.cc:276] 0320 15:27:39.639175 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.101946 (+2462771us) service_pool.cc:225] Handling call
0320 15:27:42.105335 (+ 3389us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.105954 (+ 619us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.118199 (+ 12245us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.118205 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:42.118287 (+ 82us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.118406 (+ 119us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.118515 (+ 109us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.112242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.112396 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.112403 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:42.112406 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:42.112513 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.112528 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.112533 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.112610 (+ 77us) tablet.cc:700] Row locks acquired
0320 15:27:42.112613 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:42.112649 (+ 36us) write_op.cc:270] Start()
0320 15:27:42.112684 (+ 35us) write_op.cc:276] Timestamp: P: 1774020462112601 usec, L: 0
0320 15:27:42.112687 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.112891 (+ 204us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.116256 (+ 3365us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.116362 (+ 106us) write_op.cc:301] APPLY: starting
0320 15:27:42.116384 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.116437 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.116469 (+ 32us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.116630 (+ 161us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.116676 (+ 46us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.116682 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:42.116774 (+ 92us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.116860 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.116885 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:42.116892 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":41,"apply.run_cpu_time_us":592,"apply.run_wall_time_us":591,"num_ops":2,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":755,"prepare.run_wall_time_us":756,"raft.queue_time_us":67,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":381,"replication_time_us":3527}]]}
I20260320 15:27:42.127962 28734 tablet_bootstrap.cc:654] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.106933 29643 ts_tablet_manager.cc:1929] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.128961 29643 log.cc:1199] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7feb8477ebf841a19977a434f365f537
I20260320 15:27:42.129422 29643 ts_tablet_manager.cc:1950] T 7feb8477ebf841a19977a434f365f537 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.129805 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 139) took 7908 ms. Trace:
I20260320 15:27:42.129941 29643 rpcz_store.cc:276] 0320 15:27:34.221728 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.221781 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:42.043889 (+7822108us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:42.043900 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.069446 (+ 25546us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.106808 (+ 37362us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.129788 (+ 22980us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7821948}
I20260320 15:27:42.130393 29643 tablet_replica.cc:333] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.130620 29643 raft_consensus.cc:2243] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.130821 29643 raft_consensus.cc:2272] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.130625 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7feb8477ebf841a19977a434f365f537 (table default.Table_fdd90d848d4846dcb37f6b50bce8156b [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
I20260320 15:27:42.132169 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_d71bcddc2804450dbb050fdc6c54338a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.133047 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d71bcddc2804450dbb050fdc6c54338a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.143034 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 101) took 2490 ms. Trace:
I20260320 15:27:42.145249 30287 tablet.cc:1723] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.150013 30289 tablet.cc:1723] T 0e3f6d9a528946908891f77f951d23dd P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.150815 30114 consensus_queue.cc:1048] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.154660 28734 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.155120 28734 ts_tablet_manager.cc:1403] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20260320 15:27:42.156203 28734 raft_consensus.cc:359] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.156481 28734 raft_consensus.cc:385] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.156599 28734 raft_consensus.cc:740] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.156849 28734 consensus_queue.cc:260] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.157655 28734 ts_tablet_manager.cc:1434] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.158126 28734 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.158761 28890 rpcz_store.cc:276] 0320 15:27:39.652854 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.131890 (+2479036us) service_pool.cc:225] Handling call
0320 15:27:42.133223 (+ 1333us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.133756 (+ 533us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.142637 (+ 8881us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.142644 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:42.142729 (+ 85us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.142860 (+ 131us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.142967 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.134349 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.134470 (+ 121us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.134477 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:42.134479 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:42.134581 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.134597 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.134603 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.134697 (+ 94us) tablet.cc:700] Row locks acquired
0320 15:27:42.134701 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:42.134749 (+ 48us) write_op.cc:270] Start()
0320 15:27:42.134791 (+ 42us) write_op.cc:276] Timestamp: P: 1774020462134692 usec, L: 0
0320 15:27:42.134795 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.135103 (+ 308us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.140712 (+ 5609us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.140813 (+ 101us) write_op.cc:301] APPLY: starting
0320 15:27:42.140833 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.140881 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.140930 (+ 49us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.141057 (+ 127us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.141095 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.141102 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:42.141188 (+ 86us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.141276 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.141301 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:42.141308 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":37,"apply.run_cpu_time_us":558,"apply.run_wall_time_us":557,"num_ops":2,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":907,"prepare.run_wall_time_us":906,"raft.queue_time_us":100,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":386,"replication_time_us":5869}]]}
I20260320 15:27:42.160068 28734 tablet_bootstrap.cc:654] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.162186 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e2cf21f098cf48c494849f6e8d339dc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.163044 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2cf21f098cf48c494849f6e8d339dc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:42.168450 28524 leader_election.cc:343] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:42.168823 28524 consensus_peers.cc:597] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7be03035dc3d44f9aa5b43b49281a940. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.170104 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.170320 28443 raft_consensus.cc:3060] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.172128 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 139) took 7984 ms. Trace:
I20260320 15:27:42.172252 29678 rpcz_store.cc:276] 0320 15:27:34.188039 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.188120 (+ 81us) service_pool.cc:225] Handling call
0320 15:27:41.904228 (+7716108us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:41.904243 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:41.984309 (+ 80066us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x616000142c54. stack: 00007f7f40429076 00007f7f40428e30 00007f7f3f3e33ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f7f4c3287e0 00007f7f4d679dc3 00007f7f5023da77 00007f7f50270b44 00007f7f50251560 00007f7f502512ed 0000000000a8b58c 00007f7f4046ef5f 00007f7f4047227b 00007f7f404720f8
0320 15:27:41.993349 (+ 9040us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.002974 (+ 9625us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.172101 (+169127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":182675328,"tablet-delete.queue_time_us":7714794}
I20260320 15:27:42.172730 29678 tablet_replica.cc:333] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.172986 29678 raft_consensus.cc:2243] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.173177 29678 raft_consensus.cc:2272] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.173238 28443 raft_consensus.cc:2468] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.173907 29678 ts_tablet_manager.cc:1916] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.177263 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet bdd8f5ccbd35418684fd27459c7a6b9f (table default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]) successfully deleted
I20260320 15:27:42.177932 28443 raft_consensus.cc:1275] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Refusing update from remote peer 3ce5383d28b347ab979765b880e5773c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260320 15:27:42.178454 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.178807 30114 consensus_queue.cc:1048] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260320 15:27:42.178978 29640 ts_tablet_manager.cc:1950] T b03e7555b9a646b599acb0c7dff90c9a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.182691 30114 consensus_queue.cc:1048] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.184509 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:42.185144 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:42.185153 30290 tablet.cc:1723] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.185546 30287 tablet.cc:1723] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.195569 28736 ts_tablet_manager.cc:1434] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.290s user 0.002s sys 0.001s
I20260320 15:27:42.196002 28736 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.197736 29643 ts_tablet_manager.cc:1916] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.197777 28736 tablet_bootstrap.cc:654] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.198190 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 102) took 2536 ms. Trace:
I20260320 15:27:42.198937 28890 rpcz_store.cc:276] 0320 15:27:39.661800 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.161896 (+2500096us) service_pool.cc:225] Handling call
0320 15:27:42.163253 (+ 1357us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.169692 (+ 6439us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.196894 (+ 27202us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.196903 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:42.197008 (+ 105us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.197131 (+ 123us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.198109 (+ 978us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.176765 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.176929 (+ 164us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.176937 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:42.176940 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:42.177057 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.177075 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.177081 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.177169 (+ 88us) tablet.cc:700] Row locks acquired
0320 15:27:42.177171 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:42.177214 (+ 43us) write_op.cc:270] Start()
0320 15:27:42.177262 (+ 48us) write_op.cc:276] Timestamp: P: 1774020462177194 usec, L: 0
0320 15:27:42.177267 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.177508 (+ 241us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.182728 (+ 5220us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.182873 (+ 145us) write_op.cc:301] APPLY: starting
0320 15:27:42.182894 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.182946 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.182985 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.183096 (+ 111us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.183133 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.183141 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:42.183222 (+ 81us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.183310 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.183336 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:42.183344 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":532,"apply.run_wall_time_us":532,"num_ops":2,"prepare.queue_time_us":6452,"prepare.run_cpu_time_us":875,"prepare.run_wall_time_us":875,"raft.queue_time_us":82,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":233,"replication_time_us":5424}]]}
I20260320 15:27:42.202003 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 122) took 8191 ms. Trace:
I20260320 15:27:42.202142 29640 rpcz_store.cc:276] 0320 15:27:34.010720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.010795 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:41.984844 (+7974049us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:41.984859 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.047333 (+ 62474us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.118794 (+ 71461us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.201971 (+ 83177us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23671,"tablet-delete.queue_time_us":7967344}
I20260320 15:27:42.202633 29640 tablet_replica.cc:333] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.202862 29640 raft_consensus.cc:2243] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.203049 29640 raft_consensus.cc:2272] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.202906 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b03e7555b9a646b599acb0c7dff90c9a (table default.Table_a894581517cb4471a20cffa8c3ec26f5 [id=e539cb674eb14059bdb2332a1d1f5bda]) successfully deleted
I20260320 15:27:42.203740 29640 ts_tablet_manager.cc:1916] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.204320 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_7abb6e0bedbe439daf8e7bbede311b1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.207223 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7abb6e0bedbe439daf8e7bbede311b1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.217093 29678 ts_tablet_manager.cc:1929] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.219213 30114 consensus_queue.cc:1048] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:42.224328 28524 consensus_peers.cc:597] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecaa08ba34514c5e99a9e2d4ba9268a9. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.224797 29678 log.cc:1199] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ecaa08ba34514c5e99a9e2d4ba9268a9
W20260320 15:27:42.224954 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 203602us
I20260320 15:27:42.225346 29678 ts_tablet_manager.cc:1950] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:42.225205 28383 leader_election.cc:343] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.225528 28383 leader_election.cc:304] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.225739 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 141) took 8030 ms. Trace:
I20260320 15:27:42.225850 29678 rpcz_store.cc:276] 0320 15:27:34.195254 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.198486 (+ 3232us) service_pool.cc:225] Handling call
0320 15:27:42.172673 (+7974187us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:42.172685 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.182572 (+ 9887us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.216965 (+ 34393us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.225719 (+ 8754us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7973952}
I20260320 15:27:42.226090 30168 raft_consensus.cc:2749] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:42.226087 28524 leader_election.cc:343] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
I20260320 15:27:42.226262 29678 tablet_replica.cc:333] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.226315 28524 leader_election.cc:304] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:42.226475 29678 raft_consensus.cc:2243] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:42.222558 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet e779a274913f4caeaf82e3f180457830 (table default.Table_80ba3729fee5441b983da0a2323b6ecb [id=599d8dbc463f427ba338330559e2591c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e779a274913f4caeaf82e3f180457830 consensus state has no leader
I20260320 15:27:42.226987 29939 raft_consensus.cc:2749] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.227324 29678 pending_rounds.cc:70] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:42.227406 29678 pending_rounds.cc:77] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387809865293824 op_type: NO_OP noop_request { }
I20260320 15:27:42.227610 29678 raft_consensus.cc:2889] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:42.227715 29678 raft_consensus.cc:2272] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.229463 28733 tablet_bootstrap.cc:492] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.229790 28733 ts_tablet_manager.cc:1403] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.278s user 0.015s sys 0.003s
I20260320 15:27:42.230708 28733 raft_consensus.cc:359] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.231441 28733 raft_consensus.cc:385] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.231685 28733 raft_consensus.cc:740] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.232314 28733 consensus_queue.cc:260] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.232910 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (table default.Table_30015eb07aba409bb03a37140eea20d6 [id=83739ddc977f4089b85ad8e6ea64df8b]) successfully deleted
I20260320 15:27:42.238780 29678 ts_tablet_manager.cc:1916] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.242143 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.242205 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.242601 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.242980 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:42.245450 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.246351 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 103) took 2578 ms. Trace:
I20260320 15:27:42.248889 28890 rpcz_store.cc:276] 0320 15:27:39.668163 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.204028 (+2535865us) service_pool.cc:225] Handling call
0320 15:27:42.226694 (+ 22666us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.229585 (+ 2891us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.245862 (+ 16277us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.245873 (+ 11us) write_op.cc:454] Released schema lock
0320 15:27:42.245985 (+ 112us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.246136 (+ 151us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.246272 (+ 136us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.236022 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.236227 (+ 205us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.236238 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:42.236243 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:42.236384 (+ 141us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.236405 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.236413 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.236520 (+ 107us) tablet.cc:700] Row locks acquired
0320 15:27:42.236522 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:42.236568 (+ 46us) write_op.cc:270] Start()
0320 15:27:42.236610 (+ 42us) write_op.cc:276] Timestamp: P: 1774020462236543 usec, L: 0
0320 15:27:42.236614 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.236850 (+ 236us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.240677 (+ 3827us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.240802 (+ 125us) write_op.cc:301] APPLY: starting
0320 15:27:42.240819 (+ 17us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.240868 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.240899 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.241020 (+ 121us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.241056 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.241064 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:42.241146 (+ 82us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.241234 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.241263 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:42.241270 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":74,"apply.run_cpu_time_us":526,"apply.run_wall_time_us":525,"num_ops":2,"prepare.queue_time_us":5691,"prepare.run_cpu_time_us":987,"prepare.run_wall_time_us":988,"raft.queue_time_us":91,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":329,"replication_time_us":4026}]]}
I20260320 15:27:42.253785 28736 tablet_bootstrap.cc:492] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.253958 28736 ts_tablet_manager.cc:1403] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20260320 15:27:42.254441 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c24e53f814fb49259ccd5a2d2054eeb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:42.254767 28736 raft_consensus.cc:359] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.254928 28736 raft_consensus.cc:385] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.254973 28736 raft_consensus.cc:740] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.255195 28736 consensus_queue.cc:260] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:42.256877 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c24e53f814fb49259ccd5a2d2054eeb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.258421 28736 ts_tablet_manager.cc:1434] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.258739 28736 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.264036 29678 ts_tablet_manager.cc:1929] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.264165 29678 log.cc:1199] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/07af28f257d745a1bb4173aad09045d9
I20260320 15:27:42.264518 29678 ts_tablet_manager.cc:1950] T 07af28f257d745a1bb4173aad09045d9 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.266464 30287 tablet.cc:1723] T ba4eb20eb0f1403282030727757031f7 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.265476 28736 tablet_bootstrap.cc:654] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:42.270689 28383 leader_election.cc:343] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.270907 28383 leader_election.cc:304] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.271373 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:42.271560 28736 tablet_bootstrap.cc:492] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.271595 29842 raft_consensus.cc:2749] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.271710 28736 ts_tablet_manager.cc:1403] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260320 15:27:42.271793 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 104) took 2574 ms. Trace:
I20260320 15:27:42.272454 28736 raft_consensus.cc:359] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.272713 28890 rpcz_store.cc:276] 0320 15:27:39.697066 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.254158 (+2557092us) service_pool.cc:225] Handling call
0320 15:27:42.257068 (+ 2910us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.262975 (+ 5907us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:42.263216 (+ 241us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.271372 (+ 8156us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.271379 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:42.271463 (+ 84us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.271623 (+ 160us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.271731 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.264686 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.264828 (+ 142us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.264839 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:42.264844 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:42.264981 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.265016 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.265025 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.265175 (+ 150us) tablet.cc:700] Row locks acquired
0320 15:27:42.265177 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:42.265214 (+ 37us) write_op.cc:270] Start()
0320 15:27:42.265250 (+ 36us) write_op.cc:276] Timestamp: P: 1774020462265188 usec, L: 0
0320 15:27:42.265253 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.265481 (+ 228us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.269896 (+ 4415us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.270031 (+ 135us) write_op.cc:301] APPLY: starting
0320 15:27:42.270053 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.270103 (+ 50us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.270139 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.270306 (+ 167us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.270360 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.270372 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:42.270480 (+ 108us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.270552 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.270578 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:42.270586 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":12388736,"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":616,"apply.run_wall_time_us":614,"num_ops":2,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":918,"prepare.run_wall_time_us":918,"raft.queue_time_us":72,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":291,"replication_time_us":4601}]]}
I20260320 15:27:42.272622 28736 raft_consensus.cc:385] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.274567 28736 raft_consensus.cc:740] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.274796 28736 consensus_queue.cc:260] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.275347 28736 ts_tablet_manager.cc:1434] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260320 15:27:42.273135 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 140) took 8079 ms. Trace:
I20260320 15:27:42.275648 29678 rpcz_store.cc:276] 0320 15:27:34.193243 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.193310 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:42.226200 (+8032890us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:42.226214 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.256935 (+ 30721us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.263920 (+ 6985us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.273118 (+ 9198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8023810}
I20260320 15:27:42.276206 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 143) took 8074 ms. Trace:
I20260320 15:27:42.276124 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 07af28f257d745a1bb4173aad09045d9 (table default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]) successfully deleted
I20260320 15:27:42.276288 29678 rpcz_store.cc:276] 0320 15:27:34.201481 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.202548 (+ 1067us) service_pool.cc:225] Handling call
0320 15:27:42.276069 (+8073521us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:42.276079 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.276189 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8071494}
I20260320 15:27:42.276718 29678 tablet_replica.cc:333] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.276909 29678 raft_consensus.cc:2243] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:42.277024 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd315b5d49104d679b03515b65ee30ac already in progress: creating tablet
I20260320 15:27:42.277174 29678 raft_consensus.cc:2272] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.277801 29678 ts_tablet_manager.cc:1916] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:42.279376 28383 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.279500 28383 leader_election.cc:304] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.279743 29842 raft_consensus.cc:2749] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.279893 28736 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.280691 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_dfac030930b54821b08b28ee38374406"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.280800 28383 leader_election.cc:343] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.281312 28736 tablet_bootstrap.cc:654] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:42.283828 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfac030930b54821b08b28ee38374406 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.240054 28733 ts_tablet_manager.cc:1434] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260320 15:27:42.284206 28733 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.272507 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:42.260875 30289 tablet.cc:1723] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.286193 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:42.286476 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:42.258507 28524 leader_election.cc:304] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.287184 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
W20260320 15:27:42.287253 28524 consensus_peers.cc:597] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07ff84dedc074dabb4e8e2d8460066e7. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.289964 28736 tablet_bootstrap.cc:492] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.290117 28736 ts_tablet_manager.cc:1403] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260320 15:27:42.292268 29939 raft_consensus.cc:2804] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
W20260320 15:27:42.294440 28524 leader_election.cc:343] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.295627 29599 consensus_queue.cc:1048] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.300024 29643 ts_tablet_manager.cc:1929] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:42.279284 30351 tablet.cc:1723] T ba4eb20eb0f1403282030727757031f7 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.306167 29939 raft_consensus.cc:697] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:42.306860 28383 leader_election.cc:304] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.307425 29939 consensus_queue.cc:237] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.307947 30168 raft_consensus.cc:2749] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:42.308509 28383 consensus_peers.cc:597] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.308903 30114 consensus_queue.cc:1048] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:42.309274 28383 leader_election.cc:343] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.309396 28383 leader_election.cc:304] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.309717 30168 raft_consensus.cc:2749] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.309916 28736 raft_consensus.cc:359] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.310142 28736 raft_consensus.cc:385] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.310205 28736 raft_consensus.cc:740] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.310405 28736 consensus_queue.cc:260] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.312477 28736 ts_tablet_manager.cc:1434] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260320 15:27:42.313760 29678 ts_tablet_manager.cc:1929] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.315416 29678 log.cc:1199] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:42.315832 29678 ts_tablet_manager.cc:1950] T 20a7164db1e0447fb424d8e46b747f52 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.325315 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 105) took 2612 ms. Trace:
I20260320 15:27:42.325626 28890 rpcz_store.cc:276] 0320 15:27:39.712944 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.275806 (+2562862us) service_pool.cc:225] Handling call
0320 15:27:42.284382 (+ 8576us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.284879 (+ 497us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.324760 (+ 39881us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.324773 (+ 13us) write_op.cc:454] Released schema lock
0320 15:27:42.324895 (+ 122us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.325063 (+ 168us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.325201 (+ 138us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.313890 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.314058 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.314068 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:42.314073 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:42.314210 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.314231 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.314239 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.314351 (+ 112us) tablet.cc:700] Row locks acquired
0320 15:27:42.314355 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:42.314404 (+ 49us) write_op.cc:270] Start()
0320 15:27:42.314450 (+ 46us) write_op.cc:276] Timestamp: P: 1774020462314370 usec, L: 0
0320 15:27:42.314456 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.314796 (+ 340us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.319281 (+ 4485us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.319442 (+ 161us) write_op.cc:301] APPLY: starting
0320 15:27:42.319464 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.319563 (+ 99us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.319609 (+ 46us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.319761 (+ 152us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.319810 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.319822 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:42.319924 (+ 102us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.320004 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.320034 (+ 30us) write_op.cc:454] Released schema lock
0320 15:27:42.320044 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":662,"apply.run_wall_time_us":669,"num_ops":2,"prepare.queue_time_us":28201,"prepare.run_cpu_time_us":1036,"prepare.run_wall_time_us":1037,"raft.queue_time_us":71,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":276,"replication_time_us":4781}]]}
I20260320 15:27:42.327430 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_db4a9c34ad40421aa3f8f70873479784"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:42.326967 28524 consensus_peers.cc:597] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bdd8f5ccbd35418684fd27459c7a6b9f. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:42.328377 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db4a9c34ad40421aa3f8f70873479784 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.328934 30289 tablet.cc:1723] T fa54115d4df74bec85970057b5c3dd51 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.334484 30153 raft_consensus.cc:493] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.334529 30289 tablet.cc:1723] T ce8595157679444b959b569dbb7d8df0 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.334658 30153 raft_consensus.cc:515] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.334903 30290 tablet.cc:1723] T ba4eb20eb0f1403282030727757031f7 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.335327 30153 leader_election.cc:290] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.338498 28733 tablet_bootstrap.cc:654] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.339942 29643 log.cc:1199] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:42.342523 30290 tablet.cc:1723] T fa54115d4df74bec85970057b5c3dd51 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.344053 28734 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.344280 28734 ts_tablet_manager.cc:1403] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.186s user 0.011s sys 0.015s
W20260320 15:27:42.344746 28240 consensus_peers.cc:597] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8241a342054240d2ac62b729cc9c9a62. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.350339 29643 ts_tablet_manager.cc:1950] T dd315b5d49104d679b03515b65ee30ac P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.357856 28734 raft_consensus.cc:359] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.358209 28734 raft_consensus.cc:385] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.358345 28734 raft_consensus.cc:740] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.358775 28734 consensus_queue.cc:260] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.359470 28734 ts_tablet_manager.cc:1434] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260320 15:27:42.364579 30344 raft_consensus.cc:493] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.365475 28734 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.367161 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 140) took 8145 ms. Trace:
I20260320 15:27:42.367990 28734 tablet_bootstrap.cc:654] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.368048 29643 rpcz_store.cc:276] 0320 15:27:34.222109 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.222157 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:42.130346 (+7908189us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:42.130355 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.201528 (+ 71173us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.299910 (+ 98382us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.367091 (+ 67181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7908001}
I20260320 15:27:42.369145 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.369130 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.369395 28297 raft_consensus.cc:2468] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.370680 29643 tablet_replica.cc:333] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.370961 29643 raft_consensus.cc:2243] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.371259 29643 raft_consensus.cc:2272] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.372125 29643 ts_tablet_manager.cc:1916] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.372540 29640 ts_tablet_manager.cc:1929] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:42.372673 29640 log.cc:1199] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/45a0e8e2ce7a454ebcab4e59a4de68dd
I20260320 15:27:42.373054 29640 ts_tablet_manager.cc:1950] T 45a0e8e2ce7a454ebcab4e59a4de68dd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.374787 30290 tablet.cc:1723] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.357097 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_07e18d8b90c34136804ef8e64f659f82 [id=90afb14544844334aff0e82fa156d6d9]). Expected version 2 got 1
I20260320 15:27:42.369580 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet dd315b5d49104d679b03515b65ee30ac (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:42.380725 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:42.381240 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:42.364777 30344 raft_consensus.cc:515] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.388945 28736 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.390327 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:42.390549 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:42.390753 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:42.390786 28736 tablet_bootstrap.cc:654] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.391486 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:42.391937 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:42.392278 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:42.392503 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 144) took 8190 ms. Trace:
I20260320 15:27:42.392179 30364 tablet.cc:1723] T ce8595157679444b959b569dbb7d8df0 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.392602 29678 rpcz_store.cc:276] 0320 15:27:34.201603 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.204404 (+ 2801us) service_pool.cc:225] Handling call
0320 15:27:42.276674 (+8072270us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:42.276683 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.285048 (+ 8365us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.313622 (+ 28574us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.392483 (+ 78861us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8071528}
I20260320 15:27:42.393139 29678 tablet_replica.cc:333] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.393374 29678 raft_consensus.cc:2243] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.393236 30351 tablet.cc:1723] T 2f5beaa1bc1e4465be34c8e86f131559 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.393560 29678 raft_consensus.cc:2272] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.393702 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:42.394153 29678 ts_tablet_manager.cc:1916] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.394795 28524 leader_election.cc:304] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:42.395110 29939 raft_consensus.cc:2804] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.395200 29939 raft_consensus.cc:493] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.394012 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:42.395354 29939 raft_consensus.cc:3060] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.395610 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:42.396461 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:42.396692 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:42.396884 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:42.397747 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:42.398092 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:42.398360 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:42.398627 28444 raft_consensus.cc:2468] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.398612 29939 raft_consensus.cc:515] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.399334 29939 leader_election.cc:290] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:42.399973 28524 consensus_peers.cc:597] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecaa08ba34514c5e99a9e2d4ba9268a9. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.400920 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.401203 29678 ts_tablet_manager.cc:1929] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.401338 29678 log.cc:1199] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:42.401705 29678 ts_tablet_manager.cc:1950] T ef09e4afdd1f4fce9caec4dcafa2c50c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.402537 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 145) took 8196 ms. Trace:
I20260320 15:27:42.402635 29678 rpcz_store.cc:276] 0320 15:27:34.206235 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.206293 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:42.393087 (+8186794us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:42.393097 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.397485 (+ 4388us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.401078 (+ 3593us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.402513 (+ 1435us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8186251}
W20260320 15:27:42.402755 28382 leader_election.cc:343] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.403232 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 142) took 8202 ms. Trace:
I20260320 15:27:42.403333 29678 rpcz_store.cc:276] 0320 15:27:34.201254 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.202625 (+ 1371us) service_pool.cc:225] Handling call
0320 15:27:42.403103 (+8200478us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:42.403114 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.403216 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8184262}
I20260320 15:27:42.403848 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.403833 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 123) took 8392 ms. Trace:
I20260320 15:27:42.404022 29678 tablet_replica.cc:333] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.404206 29678 raft_consensus.cc:2243] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:42.404448 28383 leader_election.cc:343] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.404524 29678 raft_consensus.cc:2272] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.404650 30344 leader_election.cc:290] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.405004 28383 leader_election.cc:304] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.405118 29678 ts_tablet_manager.cc:1916] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.405545 30344 raft_consensus.cc:2749] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.406157 28733 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.406316 28733 ts_tablet_manager.cc:1403] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.122s user 0.002s sys 0.002s
I20260320 15:27:42.407102 28733 raft_consensus.cc:359] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.407290 28733 raft_consensus.cc:385] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.407348 28733 raft_consensus.cc:740] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.407626 28733 consensus_queue.cc:260] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.408128 28733 ts_tablet_manager.cc:1434] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:42.408443 28733 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.408620 28736 tablet_bootstrap.cc:492] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.408778 28736 ts_tablet_manager.cc:1403] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260320 15:27:42.408921 30114 raft_consensus.cc:493] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.409063 30114 raft_consensus.cc:515] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.409785 30114 leader_election.cc:290] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.410315 28733 tablet_bootstrap.cc:654] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.410187 28736 raft_consensus.cc:359] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.410392 28736 raft_consensus.cc:385] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.410454 28736 raft_consensus.cc:740] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.410698 28736 consensus_queue.cc:260] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.411285 28736 ts_tablet_manager.cc:1434] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:42.411243 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.412122 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0e7ae1ff302428abf6cc592bbca0aa9" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.412801 28239 leader_election.cc:304] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:42.413139 30114 raft_consensus.cc:2749] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.413488 28736 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.415454 28736 tablet_bootstrap.cc:654] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.420329 30114 consensus_queue.cc:1048] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.426091 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.426347 28302 raft_consensus.cc:3060] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.427628 30287 tablet.cc:1723] T ce8595157679444b959b569dbb7d8df0 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.428785 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec6961acec774d2f9161f057f64b3fdf" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.429177 28446 raft_consensus.cc:3060] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.429205 28302 raft_consensus.cc:2468] T ec6961acec774d2f9161f057f64b3fdf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.403986 29640 rpcz_store.cc:276] 0320 15:27:34.011077 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.011149 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:42.202567 (+8191418us) ts_tablet_manager.cc:1151] Deleting tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
0320 15:27:42.202582 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.325711 (+123129us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.372423 (+ 46712us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.403780 (+ 31357us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8169730}
I20260320 15:27:42.443075 30287 tablet.cc:1723] T fa54115d4df74bec85970057b5c3dd51 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:42.443846 28524 leader_election.cc:304] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:42.444422 30155 raft_consensus.cc:2804] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:42.444551 30155 raft_consensus.cc:697] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:42.444401 30364 tablet.cc:1723] T 2f5beaa1bc1e4465be34c8e86f131559 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.444818 30155 consensus_queue.cc:237] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.449808 28736 tablet_bootstrap.cc:492] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.450891 29640 tablet_replica.cc:333] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.451136 29640 raft_consensus.cc:2243] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:42.451393 28736 ts_tablet_manager.cc:1403] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260320 15:27:42.451524 29640 raft_consensus.cc:2272] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.452615 29640 ts_tablet_manager.cc:1916] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.398834 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 20a7164db1e0447fb424d8e46b747f52 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]) successfully deleted
I20260320 15:27:42.418635 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 106) took 2697 ms. Trace:
I20260320 15:27:42.440688 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 45a0e8e2ce7a454ebcab4e59a4de68dd (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]) successfully deleted
W20260320 15:27:42.453323 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 54465.8us
I20260320 15:27:42.453616 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ef09e4afdd1f4fce9caec4dcafa2c50c (table default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]) successfully deleted
W20260320 15:27:42.453888 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 56fa691a26114657ad768545a30dfd3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56fa691a26114657ad768545a30dfd3c already in progress: creating tablet
I20260320 15:27:42.453431 28890 rpcz_store.cc:276] 0320 15:27:39.721003 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.327092 (+2606089us) service_pool.cc:225] Handling call
0320 15:27:42.380190 (+ 53098us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.380738 (+ 548us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.388446 (+ 7708us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.388565 (+ 119us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.418553 (+ 29988us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.381426 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.381617 (+ 191us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.381627 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:42.381632 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:42.381763 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.381786 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.381793 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.381907 (+ 114us) tablet.cc:700] Row locks acquired
0320 15:27:42.381911 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:42.381957 (+ 46us) write_op.cc:270] Start()
0320 15:27:42.382005 (+ 48us) write_op.cc:276] Timestamp: P: 1774020462381917 usec, L: 0
0320 15:27:42.382012 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.382230 (+ 218us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.386782 (+ 4552us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.386945 (+ 163us) write_op.cc:301] APPLY: starting
0320 15:27:42.386967 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.387032 (+ 65us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.387068 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.387169 (+ 101us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.387203 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.387215 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:42.387310 (+ 95us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.387382 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.387404 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:42.387410 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:42.387488 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.387492 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":532,"apply.run_wall_time_us":530,"num_ops":2,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":935,"prepare.run_wall_time_us":937,"raft.queue_time_us":72,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":281,"replication_time_us":4725}]]}
I20260320 15:27:42.455242 30344 raft_consensus.cc:493] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.455382 30344 raft_consensus.cc:515] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.455685 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:42.456126 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:42.456730 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.456066 30344 leader_election.cc:290] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.456974 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:42.457346 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
W20260320 15:27:42.457593 28382 leader_election.cc:343] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.457551 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.457885 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:42.457985 30287 tablet.cc:1723] T 2f5beaa1bc1e4465be34c8e86f131559 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:42.458094 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:42.456151 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c30a162e7c4c49db8ef902e38586b19a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:42.458426 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
W20260320 15:27:42.458731 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c30a162e7c4c49db8ef902e38586b19a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.452567 28736 raft_consensus.cc:359] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.461061 28736 raft_consensus.cc:385] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.461196 28736 raft_consensus.cc:740] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.461493 28736 consensus_queue.cc:260] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.462790 28736 ts_tablet_manager.cc:1434] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260320 15:27:42.462954 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:42.464655 28736 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:42.470893 28383 leader_election.cc:343] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.471076 28383 leader_election.cc:304] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.471882 30344 raft_consensus.cc:2749] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.483786 28446 raft_consensus.cc:2468] T ec6961acec774d2f9161f057f64b3fdf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.484047 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 107) took 2746 ms. Trace:
I20260320 15:27:42.484225 28890 rpcz_store.cc:276] 0320 15:27:39.737487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.455898 (+2718411us) service_pool.cc:225] Handling call
0320 15:27:42.458904 (+ 3006us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.459465 (+ 561us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.483720 (+ 24255us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.483851 (+ 131us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.483980 (+ 129us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.462984 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.463143 (+ 159us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.463152 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:42.463155 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:42.463264 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.463282 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.463288 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.463378 (+ 90us) tablet.cc:700] Row locks acquired
0320 15:27:42.463380 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:42.463419 (+ 39us) write_op.cc:270] Start()
0320 15:27:42.463466 (+ 47us) write_op.cc:276] Timestamp: P: 1774020462463374 usec, L: 0
0320 15:27:42.463471 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.463731 (+ 260us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.467422 (+ 3691us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.467582 (+ 160us) write_op.cc:301] APPLY: starting
0320 15:27:42.467603 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.467654 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.467691 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.467809 (+ 118us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.467858 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.467869 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:42.467959 (+ 90us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.468026 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.468051 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:42.468058 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:42.468130 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.468134 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":102,"apply.run_cpu_time_us":539,"apply.run_wall_time_us":538,"num_ops":2,"prepare.queue_time_us":860,"prepare.run_cpu_time_us":875,"prepare.run_wall_time_us":876,"raft.queue_time_us":43,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":286,"replication_time_us":3911,"spinlock_wait_cycles":768}]]}
I20260320 15:27:42.485356 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_f832b79a92ff48718c372c5aee5ce446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:42.486011 28733 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.486188 28733 ts_tablet_manager.cc:1403] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.078s user 0.004s sys 0.000s
W20260320 15:27:42.486178 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f832b79a92ff48718c372c5aee5ce446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:42.487061 28733 raft_consensus.cc:359] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.487275 28733 raft_consensus.cc:385] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.487341 28733 raft_consensus.cc:740] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.487632 28733 consensus_queue.cc:260] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:42.488831 28524 consensus_peers.cc:597] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc009401bb3d440f9d36e080efef1204. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.489899 28736 tablet_bootstrap.cc:654] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.496483 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:42.496801 28734 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.496964 28734 ts_tablet_manager.cc:1403] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.132s user 0.010s sys 0.002s
I20260320 15:27:42.498404 28734 raft_consensus.cc:359] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.498682 28734 raft_consensus.cc:385] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.498751 28734 raft_consensus.cc:740] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.499050 28734 consensus_queue.cc:260] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.499831 28734 ts_tablet_manager.cc:1434] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260320 15:27:42.500231 28734 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.502363 28734 tablet_bootstrap.cc:654] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.505735 29643 ts_tablet_manager.cc:1929] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.506598 29643 log.cc:1199] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/bdd8f5ccbd35418684fd27459c7a6b9f
W20260320 15:27:42.508733 28524 consensus_peers.cc:597] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.509500 29939 consensus_queue.cc:1048] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:42.510015 28524 consensus_peers.cc:597] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc009401bb3d440f9d36e080efef1204. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.511003 29643 ts_tablet_manager.cc:1950] T bdd8f5ccbd35418684fd27459c7a6b9f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.515647 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 141) took 8292 ms. Trace:
I20260320 15:27:42.518699 29643 rpcz_store.cc:276] 0320 15:27:34.222494 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.222776 (+ 282us) service_pool.cc:225] Handling call
0320 15:27:42.370577 (+8147801us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:42.370591 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.436214 (+ 65623us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.504913 (+ 68699us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.515384 (+ 10471us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8147231}
I20260320 15:27:42.521325 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet bdd8f5ccbd35418684fd27459c7a6b9f (table default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]) successfully deleted
I20260320 15:27:42.522208 29643 tablet_replica.cc:333] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.522358 28736 tablet_bootstrap.cc:492] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.522518 28736 ts_tablet_manager.cc:1403] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.001s
I20260320 15:27:42.523931 29640 ts_tablet_manager.cc:1929] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.524070 29640 log.cc:1199] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:42.524564 29640 ts_tablet_manager.cc:1950] T 1bf5e17dea5445d9909a229fa8315348 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.524952 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 125) took 8512 ms. Trace:
I20260320 15:27:42.525049 29640 rpcz_store.cc:276] 0320 15:27:34.012259 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.012315 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:42.440492 (+8428177us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:42.450810 (+ 10318us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.519884 (+ 69074us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.523792 (+ 3908us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.524929 (+ 1137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8407325}
I20260320 15:27:42.525419 29640 tablet_replica.cc:333] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.525750 28733 ts_tablet_manager.cc:1434] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260320 15:27:42.526072 28733 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.526039 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 108) took 2674 ms. Trace:
I20260320 15:27:42.526116 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 1bf5e17dea5445d9909a229fa8315348 (table default.Table_7a91c52621c94498bc3d4d579569893c [id=44b5fd7173fc4a49a74c9de72482d373]) successfully deleted
I20260320 15:27:42.526167 28890 rpcz_store.cc:276] 0320 15:27:39.851795 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.485075 (+2633280us) service_pool.cc:225] Handling call
0320 15:27:42.486346 (+ 1271us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:42.487120 (+ 774us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:42.525721 (+ 38601us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:42.525854 (+ 133us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:42.525955 (+ 101us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.519753 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.519955 (+ 202us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.519965 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:42.519970 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:42.520105 (+ 135us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.520127 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.520135 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.520263 (+ 128us) tablet.cc:700] Row locks acquired
0320 15:27:42.520268 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:42.520320 (+ 52us) write_op.cc:270] Start()
0320 15:27:42.520369 (+ 49us) write_op.cc:276] Timestamp: P: 1774020462520266 usec, L: 0
0320 15:27:42.520375 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.520653 (+ 278us) log.cc:844] Serialized 433 byte log entry
0320 15:27:42.524799 (+ 4146us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.524938 (+ 139us) write_op.cc:301] APPLY: starting
0320 15:27:42.524958 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.525008 (+ 50us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.525037 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.525155 (+ 118us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.525190 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:42.525198 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:42.525283 (+ 85us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.525361 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.525389 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:42.525397 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:42.525485 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.525488 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":531,"apply.run_wall_time_us":531,"num_ops":2,"prepare.queue_time_us":31703,"prepare.run_cpu_time_us":1049,"prepare.run_wall_time_us":1050,"raft.queue_time_us":39,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":291,"replication_time_us":4389,"spinlock_wait_cycles":44160}]]}
I20260320 15:27:42.526902 28890 master_service.cc:954] ReplaceTablet: received request to replace tablet 591ccc1d349e4fd599d5e343f91defe2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:42.527351 28890 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 109) took 2629 ms. Trace:
I20260320 15:27:42.527470 28890 rpcz_store.cc:276] 0320 15:27:39.897591 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.526802 (+2629211us) service_pool.cc:225] Handling call
0320 15:27:42.527312 (+ 510us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:42.527837 28890 master_service.cc:954] ReplaceTablet: received request to replace tablet aba79ddde6e842dfb67597c5d9e371cb from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:42.528688 28736 raft_consensus.cc:359] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.528920 28736 raft_consensus.cc:385] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.529003 28736 raft_consensus.cc:740] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.529634 28736 consensus_queue.cc:260] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.530205 28736 ts_tablet_manager.cc:1434] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:42.529676 29643 raft_consensus.cc:2243] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.531994 28736 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.531237 29643 raft_consensus.cc:2272] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.539453 28890 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet aba79ddde6e842dfb67597c5d9e371cb
I20260320 15:27:42.539378 29643 ts_tablet_manager.cc:1916] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.540161 28890 catalog_manager.cc:6741] ReplaceTablet: tablet aba79ddde6e842dfb67597c5d9e371cb deleted and replaced by tablet 71db28879e554f578b8a7b0e14419ab7
I20260320 15:27:42.540356 28890 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 110) took 2386 ms. Trace:
I20260320 15:27:42.540570 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet aba79ddde6e842dfb67597c5d9e371cb with delete_type TABLET_DATA_DELETED (replaced by tablet 71db28879e554f578b8a7b0e14419ab7) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:42.540472 28890 rpcz_store.cc:276] 0320 15:27:40.153470 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.527768 (+2374298us) service_pool.cc:225] Handling call
0320 15:27:42.540296 (+ 12528us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.534071 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.534231 (+ 160us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.534239 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:42.534242 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:42.534372 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.534390 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.534396 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.534477 (+ 81us) tablet.cc:700] Row locks acquired
0320 15:27:42.534479 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:42.534521 (+ 42us) write_op.cc:270] Start()
0320 15:27:42.534560 (+ 39us) write_op.cc:276] Timestamp: P: 1774020462534468 usec, L: 0
0320 15:27:42.534563 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.534793 (+ 230us) log.cc:844] Serialized 622 byte log entry
0320 15:27:42.538494 (+ 3701us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.538604 (+ 110us) write_op.cc:301] APPLY: starting
0320 15:27:42.538619 (+ 15us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.538666 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.538709 (+ 43us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.538813 (+ 104us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.538873 (+ 60us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:42.538881 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:42.538956 (+ 75us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.539032 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.539053 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:42.539059 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:42.539135 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.539139 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":61,"apply.run_cpu_time_us":518,"apply.run_wall_time_us":517,"num_ops":2,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":868,"prepare.run_wall_time_us":869,"raft.queue_time_us":39,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":317,"replication_time_us":3895}]]}
I20260320 15:27:42.541247 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet aba79ddde6e842dfb67597c5d9e371cb with delete_type TABLET_DATA_DELETED (replaced by tablet 71db28879e554f578b8a7b0e14419ab7) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:42.541427 28890 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_78b650f7f9354416913f3c232c22eb59" } new_table_name: "default.Table_303ce9b5065e40fcaf9800c531b31dee" modify_external_catalogs: true
I20260320 15:27:42.542680 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet aba79ddde6e842dfb67597c5d9e371cb with delete_type TABLET_DATA_DELETED (replaced by tablet 71db28879e554f578b8a7b0e14419ab7) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:42.544473 28736 tablet_bootstrap.cc:654] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.551455 30114 raft_consensus.cc:493] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.551659 30114 raft_consensus.cc:515] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.552307 30114 leader_election.cc:290] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.552965 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.556327 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0acad11980f842e7b3a09216241e493d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.559031 28239 leader_election.cc:304] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:42.559296 29678 ts_tablet_manager.cc:1929] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.559365 30114 raft_consensus.cc:2749] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.559521 29678 log.cc:1199] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:42.560099 29678 ts_tablet_manager.cc:1950] T 0e9a4e61bab44d768d9adfed00c945e9 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.560468 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 147) took 8318 ms. Trace:
I20260320 15:27:42.560559 29678 rpcz_store.cc:276] 0320 15:27:34.242069 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.242179 (+ 110us) service_pool.cc:225] Handling call
0320 15:27:42.403979 (+8161800us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:42.403987 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.504838 (+100851us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.559071 (+ 54233us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.560451 (+ 1380us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8149914}
I20260320 15:27:42.564637 28303 raft_consensus.cc:3060] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.564739 30313 raft_consensus.cc:493] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.564908 30313 raft_consensus.cc:515] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.565691 30313 leader_election.cc:290] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.566084 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.566836 28382 leader_election.cc:343] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.567239 29678 tablet_replica.cc:333] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.567443 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.567600 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0e9a4e61bab44d768d9adfed00c945e9 (table default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]) successfully deleted
W20260320 15:27:42.568174 28383 leader_election.cc:343] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.568329 28383 leader_election.cc:304] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.568472 28734 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.568615 28734 ts_tablet_manager.cc:1403] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.068s user 0.003s sys 0.000s
I20260320 15:27:42.568789 30313 raft_consensus.cc:2749] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.569343 28734 raft_consensus.cc:359] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.569572 28734 raft_consensus.cc:385] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.569667 28734 raft_consensus.cc:740] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.569936 28734 consensus_queue.cc:260] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.570506 28734 ts_tablet_manager.cc:1434] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:42.572434 29678 raft_consensus.cc:2243] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.572394 29939 consensus_queue.cc:1048] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.573482 28736 tablet_bootstrap.cc:492] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.573668 28736 ts_tablet_manager.cc:1403] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.002s
I20260320 15:27:42.565482 28733 tablet_bootstrap.cc:654] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.579366 28734 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.579339 29678 raft_consensus.cc:2272] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.580161 29678 ts_tablet_manager.cc:1916] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.580298 28736 raft_consensus.cc:359] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.580502 28736 raft_consensus.cc:385] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.580577 28736 raft_consensus.cc:740] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.580863 28736 consensus_queue.cc:260] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.581014 28734 tablet_bootstrap.cc:654] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.581496 28736 ts_tablet_manager.cc:1434] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260320 15:27:42.581840 28736 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.584236 28736 tablet_bootstrap.cc:654] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.592985 30313 raft_consensus.cc:493] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.593152 30313 raft_consensus.cc:515] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.593784 30313 leader_election.cc:290] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.594460 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4a30476e7146afae1e77dad078136e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.595264 28382 leader_election.cc:343] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.595631 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:42.595974 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad4a30476e7146afae1e77dad078136e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:42.596413 28383 leader_election.cc:343] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.596529 28383 leader_election.cc:304] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.597808 30344 raft_consensus.cc:2749] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.598574 28734 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.598713 28734 ts_tablet_manager.cc:1403] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260320 15:27:42.599391 28734 raft_consensus.cc:359] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.599601 28734 raft_consensus.cc:385] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.599664 28734 raft_consensus.cc:740] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.599884 28734 consensus_queue.cc:260] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.600376 28734 ts_tablet_manager.cc:1434] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.600677 28734 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.602512 28734 tablet_bootstrap.cc:654] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.606542 29678 ts_tablet_manager.cc:1929] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.606770 29678 log.cc:1199] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/757b5afdea674217909610e3719a8804
I20260320 15:27:42.607190 29678 ts_tablet_manager.cc:1950] T 757b5afdea674217909610e3719a8804 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.608304 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 146) took 8390 ms. Trace:
I20260320 15:27:42.608430 29678 rpcz_store.cc:276] 0320 15:27:34.218307 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.218367 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:42.567174 (+8348807us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:42.567188 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.594830 (+ 27642us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.606357 (+ 11527us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.608276 (+ 1919us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8312810}
I20260320 15:27:42.609063 29678 tablet_replica.cc:333] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.610632 29640 raft_consensus.cc:2243] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.609026 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 757b5afdea674217909610e3719a8804 (table default.Table_38cee1701ad24626b2929b539fdb3c3e [id=e24b82becde543b8ad89b3b5ed7cc08b]) successfully deleted
I20260320 15:27:42.610865 29640 raft_consensus.cc:2272] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.610965 29678 raft_consensus.cc:2243] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:42.611398 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:42.611459 29678 raft_consensus.cc:2272] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.611500 29640 ts_tablet_manager.cc:1916] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.612112 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:42.612197 29678 ts_tablet_manager.cc:1916] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.612478 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:42.613386 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:42.621268 29678 ts_tablet_manager.cc:1929] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.622305 28734 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.622536 28734 ts_tablet_manager.cc:1403] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260320 15:27:42.623385 28734 raft_consensus.cc:359] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.623749 28734 raft_consensus.cc:385] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.623837 28734 raft_consensus.cc:740] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.624114 28734 consensus_queue.cc:260] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.624878 28734 ts_tablet_manager.cc:1434] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.625272 28734 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.627401 28736 tablet_bootstrap.cc:492] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.627596 28736 ts_tablet_manager.cc:1403] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.000s
I20260320 15:27:42.627832 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:42.628226 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7a8e822249154a1784ebd66f8e56abdf (table default.Table_07e18d8b90c34136804ef8e64f659f82 [id=90afb14544844334aff0e82fa156d6d9]). Expected version 2 got 1
I20260320 15:27:42.628511 28736 raft_consensus.cc:359] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.628633 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:42.628696 28736 raft_consensus.cc:385] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.628774 28736 raft_consensus.cc:740] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.628933 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:42.627005 28734 tablet_bootstrap.cc:654] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.629541 28733 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.630005 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:42.629696 28733 ts_tablet_manager.cc:1403] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.104s user 0.003s sys 0.001s
I20260320 15:27:42.630836 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:42.631038 28733 raft_consensus.cc:359] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.631250 28733 raft_consensus.cc:385] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.631318 28733 raft_consensus.cc:740] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.631705 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:42.631646 28733 consensus_queue.cc:260] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.632182 28733 ts_tablet_manager.cc:1434] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:42.632395 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:42.632488 28733 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.633106 28901 catalog_manager.cc:5671] T 60d0b36f462945cbbca7e389edf494d3 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:42.633852 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:42.634125 28733 tablet_bootstrap.cc:654] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.634337 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:42.634827 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:42.635771 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:42.636104 29678 log.cc:1199] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:42.636703 29678 ts_tablet_manager.cc:1950] T 43a0fadc7dac4e21ad034e4cd2d70826 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.638041 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:42.639214 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 148) took 8393 ms. Trace:
I20260320 15:27:42.639328 29678 rpcz_store.cc:276] 0320 15:27:34.245597 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.245660 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:42.609010 (+8363350us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:42.609022 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.617956 (+ 8934us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.621147 (+ 3191us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.639187 (+ 18040us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8351529}
I20260320 15:27:42.641199 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:42.641752 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 43a0fadc7dac4e21ad034e4cd2d70826 (table default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]) successfully deleted
I20260320 15:27:42.641566 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:42.642326 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:42.642859 28734 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.643059 28734 ts_tablet_manager.cc:1403] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260320 15:27:42.643934 28734 raft_consensus.cc:359] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.644130 28734 raft_consensus.cc:385] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.644254 28734 raft_consensus.cc:740] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.644222 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:42.644563 28734 consensus_queue.cc:260] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.645170 28734 ts_tablet_manager.cc:1434] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.646106 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
W20260320 15:27:42.646515 28524 consensus_peers.cc:597] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef09e4afdd1f4fce9caec4dcafa2c50c. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.647858 29678 tablet_replica.cc:333] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.648494 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:42.648447 29678 raft_consensus.cc:2243] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.648758 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:42.648973 29678 raft_consensus.cc:2272] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.649475 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
I20260320 15:27:42.649763 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:42.650640 29678 ts_tablet_manager.cc:1916] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.651441 29643 ts_tablet_manager.cc:1929] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.650027 28901 catalog_manager.cc:5671] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:42.652462 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:42.659287 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:42.660172 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:42.660843 29640 ts_tablet_manager.cc:1929] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.661152 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:42.661221 28736 consensus_queue.cc:260] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.661499 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:42.661923 28736 ts_tablet_manager.cc:1434] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.034s user 0.002s sys 0.000s
I20260320 15:27:42.662245 28736 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.664151 28736 tablet_bootstrap.cc:654] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.661942 28901 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:42.656687 28734 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.672302 29643 log.cc:1199] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:42.672750 29643 ts_tablet_manager.cc:1950] T 20a7164db1e0447fb424d8e46b747f52 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.673837 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 142) took 8450 ms. Trace:
I20260320 15:27:42.673905 28734 tablet_bootstrap.cc:654] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.673935 29643 rpcz_store.cc:276] 0320 15:27:34.223073 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.223296 (+ 223us) service_pool.cc:225] Handling call
0320 15:27:42.521809 (+8298513us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:42.521821 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.631384 (+109563us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.651324 (+ 19940us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.673808 (+ 22484us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8297996}
I20260320 15:27:42.674311 29643 tablet_replica.cc:333] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.674427 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 20a7164db1e0447fb424d8e46b747f52 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]) successfully deleted
I20260320 15:27:42.674510 29643 raft_consensus.cc:2243] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.674667 29643 raft_consensus.cc:2272] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.675256 29643 ts_tablet_manager.cc:1916] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.679582 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:42.688061 28733 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.688319 28733 ts_tablet_manager.cc:1403] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.000s
I20260320 15:27:42.688406 30313 raft_consensus.cc:493] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.688580 30313 raft_consensus.cc:515] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.689265 28733 raft_consensus.cc:359] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.689435 28733 raft_consensus.cc:385] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.689447 30313 leader_election.cc:290] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.689486 28733 raft_consensus.cc:740] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.689697 28733 consensus_queue.cc:260] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.690299 28733 ts_tablet_manager.cc:1434] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:42.690230 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.690640 28733 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.690600 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.691751 28383 leader_election.cc:343] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.691887 30355 raft_consensus.cc:493] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.692003 30355 raft_consensus.cc:515] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.692566 30355 leader_election.cc:290] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.693233 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.693416 28297 raft_consensus.cc:2468] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:42.693523 28382 leader_election.cc:343] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.693553 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.693636 28382 leader_election.cc:304] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.693750 28446 raft_consensus.cc:2468] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.694294 28524 leader_election.cc:304] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:42.695173 30355 raft_consensus.cc:2804] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.695273 30355 raft_consensus.cc:493] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.695335 30355 raft_consensus.cc:3060] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.698233 30355 raft_consensus.cc:515] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.699249 30313 raft_consensus.cc:2749] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.700789 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.700979 28446 raft_consensus.cc:3060] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.704382 28446 raft_consensus.cc:2468] T e5479578a7cd470a9a4851d252a9e93e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.704984 28524 leader_election.cc:304] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.705319 30155 raft_consensus.cc:2804] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:42.705436 30155 raft_consensus.cc:697] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:42.705760 30155 consensus_queue.cc:237] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.707861 30355 leader_election.cc:290] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.708189 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5479578a7cd470a9a4851d252a9e93e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.708405 28302 raft_consensus.cc:3060] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.699107 28733 tablet_bootstrap.cc:654] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.711154 28302 raft_consensus.cc:2468] T e5479578a7cd470a9a4851d252a9e93e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.699179 29640 log.cc:1199] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:42.711843 30114 raft_consensus.cc:493] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.711977 30114 raft_consensus.cc:515] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.712280 29640 ts_tablet_manager.cc:1950] T ea6ff59d2ffe4d579e6db2375eb11414 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.712668 30114 leader_election.cc:290] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.713181 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 127) took 8699 ms. Trace:
I20260320 15:27:42.713264 29640 rpcz_store.cc:276] 0320 15:27:34.071639 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.071699 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:42.525362 (+8453663us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:42.525373 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.656161 (+130788us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.660722 (+ 4561us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.713159 (+ 52437us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":84957,"tablet-delete.queue_time_us":8450792}
I20260320 15:27:42.713696 29640 tablet_replica.cc:333] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.713757 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ea6ff59d2ffe4d579e6db2375eb11414 (table default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]) successfully deleted
I20260320 15:27:42.713928 29640 raft_consensus.cc:2243] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.714083 29640 raft_consensus.cc:2272] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.714059 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.714649 29640 ts_tablet_manager.cc:1916] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.716106 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec63fb080545de838803033f204fab" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.716892 28446 raft_consensus.cc:2393] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:42.717777 28240 leader_election.cc:304] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:42.718423 30114 raft_consensus.cc:3060] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.721194 30114 raft_consensus.cc:2749] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.722841 28575 raft_consensus.cc:3060] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.732085 30256 consensus_queue.cc:1048] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.762753 29643 ts_tablet_manager.cc:1929] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.762912 29643 log.cc:1199] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:42.763311 29643 ts_tablet_manager.cc:1950] T ef09e4afdd1f4fce9caec4dcafa2c50c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.763651 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 143) took 8540 ms. Trace:
I20260320 15:27:42.763727 29643 rpcz_store.cc:276] 0320 15:27:34.223240 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.223788 (+ 548us) service_pool.cc:225] Handling call
0320 15:27:42.674253 (+8450465us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:42.674270 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.717647 (+ 43377us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.761155 (+ 43508us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.763634 (+ 2479us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8450241}
I20260320 15:27:42.764071 29643 tablet_replica.cc:333] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.764243 29643 raft_consensus.cc:2243] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.764401 29643 raft_consensus.cc:2272] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.766330 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ef09e4afdd1f4fce9caec4dcafa2c50c (table default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]) successfully deleted
I20260320 15:27:42.769017 28736 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.769191 29640 ts_tablet_manager.cc:1929] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:42.769321 29640 log.cc:1199] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/889d620c311a42039c22af53ad0802f2
I20260320 15:27:42.769255 28736 ts_tablet_manager.cc:1403] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.005s
I20260320 15:27:42.769735 29640 ts_tablet_manager.cc:1950] T 889d620c311a42039c22af53ad0802f2 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.770092 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 129) took 8697 ms. Trace:
I20260320 15:27:42.770206 29640 rpcz_store.cc:276] 0320 15:27:34.072235 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.072278 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:42.713644 (+8641366us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:42.713655 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.741433 (+ 27778us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.769086 (+ 27653us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.770049 (+ 963us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8638997}
I20260320 15:27:42.770622 29640 tablet_replica.cc:333] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.770861 29640 raft_consensus.cc:2243] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:42.771257 29640 raft_consensus.cc:2272] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.772190 28736 raft_consensus.cc:359] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.772473 28736 raft_consensus.cc:385] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.772572 28736 raft_consensus.cc:740] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.772840 28736 consensus_queue.cc:260] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.774246 28736 ts_tablet_manager.cc:1434] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:42.774582 28736 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.775117 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:42.776242 28733 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.776417 28733 ts_tablet_manager.cc:1403] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.086s user 0.004s sys 0.003s
I20260320 15:27:42.776640 28736 tablet_bootstrap.cc:654] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.777266 28733 raft_consensus.cc:359] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.777480 28733 raft_consensus.cc:385] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.777583 28733 raft_consensus.cc:740] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.777863 28733 consensus_queue.cc:260] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.778494 28733 ts_tablet_manager.cc:1434] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:42.779045 28734 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.779202 28734 ts_tablet_manager.cc:1403] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.123s user 0.025s sys 0.016s
I20260320 15:27:42.779943 28734 raft_consensus.cc:359] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.780110 28734 raft_consensus.cc:385] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.780156 28734 raft_consensus.cc:740] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.780365 28734 consensus_queue.cc:260] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.780820 28734 ts_tablet_manager.cc:1434] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:42.781083 28734 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.782480 28734 tablet_bootstrap.cc:654] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.786144 28733 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.788084 28733 tablet_bootstrap.cc:654] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.789866 29643 ts_tablet_manager.cc:1916] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.798820 30155 raft_consensus.cc:493] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.799067 30155 raft_consensus.cc:515] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.799888 30155 leader_election.cc:290] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.800333 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.800542 28446 raft_consensus.cc:2468] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.800668 28734 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.800787 28734 ts_tablet_manager.cc:1403] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.004s
I20260320 15:27:42.800958 28524 leader_election.cc:304] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.801522 30155 raft_consensus.cc:2804] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.801563 28734 raft_consensus.cc:359] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.801658 30155 raft_consensus.cc:493] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.801712 28734 raft_consensus.cc:385] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.801762 30155 raft_consensus.cc:3060] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.801774 28734 raft_consensus.cc:740] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.802000 28734 consensus_queue.cc:260] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.802619 28734 ts_tablet_manager.cc:1434] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:42.802894 28734 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.804389 28734 tablet_bootstrap.cc:654] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.804854 30155 raft_consensus.cc:515] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.800619 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.794551 28736 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:42.805828 28736 ts_tablet_manager.cc:1403] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.002s
W20260320 15:27:42.806007 28524 leader_election.cc:343] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.806454 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.806556 28736 raft_consensus.cc:359] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.806638 28446 raft_consensus.cc:3060] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.806697 28736 raft_consensus.cc:385] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.806739 28736 raft_consensus.cc:740] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.806949 28736 consensus_queue.cc:260] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.807858 28736 ts_tablet_manager.cc:1434] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:42.810448 28446 raft_consensus.cc:2468] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.811717 28524 leader_election.cc:304] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.812058 30355 raft_consensus.cc:2804] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:42.812084 30313 raft_consensus.cc:493] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.812192 30355 raft_consensus.cc:697] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:42.812243 30313 raft_consensus.cc:515] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.812508 30355 consensus_queue.cc:237] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.813022 30313 leader_election.cc:290] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.813797 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:42.814479 28383 leader_election.cc:343] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.814586 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.815164 28382 leader_election.cc:343] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.815305 28382 leader_election.cc:304] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.815829 30313 raft_consensus.cc:2749] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.820523 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ccbf4f32636463f8e1e005a629b92cc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.820101 30155 leader_election.cc:290] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.828418 28736 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:42.832630 29643 ts_tablet_manager.cc:1929] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.832810 29643 log.cc:1199] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:42.833181 29643 ts_tablet_manager.cc:1950] T fa5618796f1a4ff3ab4c1209dad0aede P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.834056 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 144) took 8609 ms. Trace:
I20260320 15:27:42.834157 29643 rpcz_store.cc:276] 0320 15:27:34.224146 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.224220 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:42.764025 (+8539805us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:42.764033 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.817814 (+ 53781us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.832507 (+ 14693us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.834030 (+ 1523us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8539621}
I20260320 15:27:42.834534 29643 tablet_replica.cc:333] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.834744 29643 raft_consensus.cc:2243] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.834707 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet fa5618796f1a4ff3ab4c1209dad0aede (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
I20260320 15:27:42.834916 29643 raft_consensus.cc:2272] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.834141 29678 ts_tablet_manager.cc:1929] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.840721 29678 log.cc:1199] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:42.843261 29640 ts_tablet_manager.cc:1916] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.845852 30155 raft_consensus.cc:493] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.846062 30155 raft_consensus.cc:515] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:42.858017 28524 leader_election.cc:343] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.858975 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.859275 28446 raft_consensus.cc:2468] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.860569 30155 leader_election.cc:290] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.861102 29643 ts_tablet_manager.cc:1916] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.861420 28733 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.861562 28733 ts_tablet_manager.cc:1403] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.076s user 0.004s sys 0.000s
I20260320 15:27:42.866197 29640 ts_tablet_manager.cc:1929] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.866333 29640 log.cc:1199] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/c3bc4dd47817490d88725127b456cc70
I20260320 15:27:42.865208 29678 ts_tablet_manager.cc:1950] T 3ed29146bff74e86a366d3c6329176f4 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:42.866925 28734 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.867389 28734 ts_tablet_manager.cc:1403] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.065s user 0.014s sys 0.018s
I20260320 15:27:42.868575 28734 raft_consensus.cc:359] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.868918 28734 raft_consensus.cc:385] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.869020 28734 raft_consensus.cc:740] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.869330 28734 consensus_queue.cc:260] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.869381 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:42.869618 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:42.869823 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:42.869980 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:42.870324 28734 ts_tablet_manager.cc:1434] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:42.870409 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:42.871031 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:42.871459 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:42.871716 28734 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.872624 29640 ts_tablet_manager.cc:1950] T c3bc4dd47817490d88725127b456cc70 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.872566 28524 leader_election.cc:304] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:42.873191 28524 consensus_peers.cc:597] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:42.873667 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.873867 28302 raft_consensus.cc:2468] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.875142 28733 raft_consensus.cc:359] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.875348 28733 raft_consensus.cc:385] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.875401 28733 raft_consensus.cc:740] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.875633 28733 consensus_queue.cc:260] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.875782 28734 tablet_bootstrap.cc:654] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.876029 29939 raft_consensus.cc:2804] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.876099 29939 raft_consensus.cc:493] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.876152 29939 raft_consensus.cc:3060] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.878279 28733 ts_tablet_manager.cc:1434] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260320 15:27:42.878613 28733 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.880861 28733 tablet_bootstrap.cc:654] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.887456 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 130) took 8815 ms. Trace:
I20260320 15:27:42.887641 29640 rpcz_store.cc:276] 0320 15:27:34.072467 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.072509 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:42.770502 (+8697993us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:42.770511 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.856362 (+ 85851us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.866072 (+ 9710us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.887422 (+ 21350us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8695818}
I20260320 15:27:42.888020 29640 tablet_replica.cc:333] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.888252 29640 raft_consensus.cc:2243] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.888438 29640 raft_consensus.cc:2272] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.888134 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet c3bc4dd47817490d88725127b456cc70 (table default.Table_a33f108177064a298842573efef5829c [id=6da31e10c1f84544a9737bfb318d1dac]) successfully deleted
I20260320 15:27:42.892270 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:42.892587 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:42.893332 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
I20260320 15:27:42.893767 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:42.894009 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:42.894639 28904 catalog_manager.cc:5671] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:42.896587 28302 raft_consensus.cc:3060] T e0e7ae1ff302428abf6cc592bbca0aa9 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.901886 30155 raft_consensus.cc:493] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:42.902046 30155 raft_consensus.cc:515] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:42.902669 30155 leader_election.cc:290] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:42.903359 29640 ts_tablet_manager.cc:1916] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.903798 29939 raft_consensus.cc:515] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.904649 30313 raft_consensus.cc:493] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.904824 30313 raft_consensus.cc:515] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:42.905045 28524 leader_election.cc:343] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:42.905520 30313 leader_election.cc:290] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.906780 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:42.907871 28383 leader_election.cc:343] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.908452 29939 leader_election.cc:290] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.912106 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:42.912591 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:42.912900 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:42.916173 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
W20260320 15:27:42.919051 28524 leader_election.cc:343] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:42.919195 28524 leader_election.cc:304] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:42.920364 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.920571 28441 raft_consensus.cc:3060] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.920539 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 149) took 8669 ms. Trace:
I20260320 15:27:42.923337 28441 raft_consensus.cc:2468] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.922519 28736 tablet_bootstrap.cc:654] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.924392 28524 leader_election.cc:304] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.924403 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408f5b4d67514838a17c180ae2cd71df" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.924604 28297 raft_consensus.cc:3060] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.924813 29939 raft_consensus.cc:2749] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:42.924978 29939 raft_consensus.cc:2804] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:42.925071 29939 raft_consensus.cc:697] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:42.925307 29939 consensus_queue.cc:237] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.927302 28297 raft_consensus.cc:2468] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:42.927713 30313 raft_consensus.cc:493] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.927877 30313 raft_consensus.cc:515] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.928505 30313 leader_election.cc:290] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.929250 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:42.932756 28383 leader_election.cc:343] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.923148 29678 rpcz_store.cc:276] 0320 15:27:34.250492 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.251917 (+ 1425us) service_pool.cc:225] Handling call
0320 15:27:42.641575 (+8389658us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:42.647719 (+ 6144us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.701966 (+ 54247us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.799261 (+ 97295us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.920099 (+120838us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8383896}
I20260320 15:27:42.934959 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 3ed29146bff74e86a366d3c6329176f4 (table default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]) successfully deleted
I20260320 15:27:42.939599 29678 tablet_replica.cc:333] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:42.941519 28734 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:42.945083 29643 ts_tablet_manager.cc:1929] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.945441 29643 log.cc:1199] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:42.945943 29643 ts_tablet_manager.cc:1950] T 0e9a4e61bab44d768d9adfed00c945e9 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:42.946362 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 145) took 8721 ms. Trace:
I20260320 15:27:42.945818 28734 ts_tablet_manager.cc:1403] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.074s user 0.002s sys 0.005s
I20260320 15:27:42.947448 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.947471 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:42.949647 30395 consensus_queue.cc:1048] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.950858 28734 raft_consensus.cc:359] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.951303 28734 raft_consensus.cc:385] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.951399 29640 ts_tablet_manager.cc:1929] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:42.951503 28734 raft_consensus.cc:740] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.952121 28734 consensus_queue.cc:260] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.946461 29643 rpcz_store.cc:276] 0320 15:27:34.224573 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.224621 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:42.834476 (+8609855us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:42.834487 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.869289 (+ 34802us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.944944 (+ 75655us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.946335 (+ 1391us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8609689}
W20260320 15:27:42.948418 28382 leader_election.cc:343] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.953243 28382 leader_election.cc:304] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:42.953809 28382 leader_election.cc:343] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.953907 28382 leader_election.cc:304] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:42.954393 30344 raft_consensus.cc:2749] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.954566 30344 raft_consensus.cc:2749] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:42.951508 29640 log.cc:1199] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:42.955178 29640 ts_tablet_manager.cc:1950] T 8241a342054240d2ac62b729cc9c9a62 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:42.955283 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0e9a4e61bab44d768d9adfed00c945e9 (table default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]) successfully deleted
I20260320 15:27:42.955612 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 131) took 8882 ms. Trace:
I20260320 15:27:42.955704 29640 rpcz_store.cc:276] 0320 15:27:34.072786 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.072838 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:42.887967 (+8815129us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:42.887979 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:42.925412 (+ 37433us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.951258 (+ 25846us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:42.955591 (+ 4333us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8813237}
I20260320 15:27:42.956032 29640 tablet_replica.cc:333] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:42.956106 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8241a342054240d2ac62b729cc9c9a62 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
I20260320 15:27:42.956944 29643 tablet_replica.cc:333] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:42.963376 29678 raft_consensus.cc:2243] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.963678 29678 raft_consensus.cc:2272] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.963790 29643 raft_consensus.cc:2243] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:42.964226 29643 raft_consensus.cc:2272] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:42.964375 29678 ts_tablet_manager.cc:1916] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.967151 29643 ts_tablet_manager.cc:1916] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:42.956207 29640 raft_consensus.cc:2243] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:42.971643 30355 raft_consensus.cc:493] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.971856 30355 raft_consensus.cc:515] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.972987 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.973209 28446 raft_consensus.cc:2468] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:42.973680 28524 leader_election.cc:304] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.973937 30395 raft_consensus.cc:2804] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.974031 30395 raft_consensus.cc:493] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.974103 30395 raft_consensus.cc:3060] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.974512 30256 raft_consensus.cc:493] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:42.974665 30256 raft_consensus.cc:515] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.975270 30256 leader_election.cc:290] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.975517 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:42.976276 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:42.976450 28446 raft_consensus.cc:2468] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:42.976853 28240 leader_election.cc:304] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.977234 30256 raft_consensus.cc:2804] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:42.977337 30256 raft_consensus.cc:493] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:42.977417 30256 raft_consensus.cc:3060] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:42.977686 28524 leader_election.cc:343] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.978065 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:42.978570 28239 leader_election.cc:343] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.978945 30355 leader_election.cc:290] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.975932 28734 ts_tablet_manager.cc:1434] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.027s user 0.003s sys 0.000s
I20260320 15:27:42.981333 30256 raft_consensus.cc:515] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.981757 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:42.982151 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:42.982582 30256 leader_election.cc:290] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:42.982599 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.982679 28733 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:42.982772 28446 raft_consensus.cc:3060] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.982811 28733 ts_tablet_manager.cc:1403] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.104s user 0.004s sys 0.010s
I20260320 15:27:42.983253 28734 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:42.983389 30395 raft_consensus.cc:515] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.983500 28733 raft_consensus.cc:359] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.983666 28733 raft_consensus.cc:385] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:42.983712 28733 raft_consensus.cc:740] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:42.983899 28733 consensus_queue.cc:260] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:42.984063 30395 leader_election.cc:290] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:42.984342 28733 ts_tablet_manager.cc:1434] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260320 15:27:42.984624 28733 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:42.985422 28446 raft_consensus.cc:2468] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:42.986115 28733 tablet_bootstrap.cc:654] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:42.986099 28240 leader_election.cc:304] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:42.987967 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:42.988037 28304 raft_consensus.cc:3060] T 01078353a3a343a3a1231245a391cf8e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:42.988682 28524 leader_election.cc:343] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:42.989030 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6292904782fd4b619483135e58b66684" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:42.989266 28446 raft_consensus.cc:3060] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:42.993321 30155 consensus_queue.cc:1048] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:42.995430 30257 raft_consensus.cc:2804] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:42.995638 30257 raft_consensus.cc:697] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:42.995906 30257 consensus_queue.cc:237] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:42.997994 28734 tablet_bootstrap.cc:654] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.004676 30355 consensus_queue.cc:1048] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.011610 29640 raft_consensus.cc:2272] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.012342 29640 ts_tablet_manager.cc:1916] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.012797 28736 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:43.013293 28736 ts_tablet_manager.cc:1403] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.185s user 0.005s sys 0.004s
I20260320 15:27:43.014732 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:43.015558 28239 leader_election.cc:343] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.016644 28733 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.016815 28733 ts_tablet_manager.cc:1403] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.001s
I20260320 15:27:43.017741 28733 raft_consensus.cc:359] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.017973 28733 raft_consensus.cc:385] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.018028 28733 raft_consensus.cc:740] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.019747 28733 consensus_queue.cc:260] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.022809 28446 raft_consensus.cc:2468] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.023557 28524 leader_election.cc:304] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.025074 30359 raft_consensus.cc:2804] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.025286 30359 raft_consensus.cc:697] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.025950 29678 ts_tablet_manager.cc:1929] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.026099 29678 log.cc:1199] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:43.026289 30359 consensus_queue.cc:237] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.026546 29678 ts_tablet_manager.cc:1950] T e7bd0616f5604f8fbb8edd8e9c230f39 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.026896 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 150) took 8746 ms. Trace:
I20260320 15:27:43.026994 29678 rpcz_store.cc:276] 0320 15:27:34.280829 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.280939 (+ 110us) service_pool.cc:225] Handling call
0320 15:27:42.937270 (+8656331us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:42.937760 (+ 490us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.016298 (+ 78538us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.019343 (+ 3045us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.026876 (+ 7533us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8652486}
I20260320 15:27:43.027366 29678 tablet_replica.cc:333] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.027608 29678 raft_consensus.cc:2243] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.027984 29678 raft_consensus.cc:2272] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.028625 29678 ts_tablet_manager.cc:1916] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.032594 28734 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.032949 28734 ts_tablet_manager.cc:1403] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.001s
I20260320 15:27:43.033511 28733 ts_tablet_manager.cc:1434] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
I20260320 15:27:43.034884 28733 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.035318 28734 raft_consensus.cc:359] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.035831 28734 raft_consensus.cc:385] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.038995 30394 consensus_queue.cc:1048] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.035970 28734 raft_consensus.cc:740] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.030135 28736 raft_consensus.cc:359] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.044454 28736 raft_consensus.cc:385] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.041500 28734 consensus_queue.cc:260] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.045982 28736 raft_consensus.cc:740] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.048034 28733 tablet_bootstrap.cc:654] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.048781 28736 consensus_queue.cc:260] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.050398 28734 ts_tablet_manager.cc:1434] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260320 15:27:43.052636 29640 ts_tablet_manager.cc:1929] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.052779 29640 log.cc:1199] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7be03035dc3d44f9aa5b43b49281a940
I20260320 15:27:43.053148 29643 ts_tablet_manager.cc:1929] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.053692 29640 ts_tablet_manager.cc:1950] T 7be03035dc3d44f9aa5b43b49281a940 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.053828 28734 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.054205 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 132) took 8981 ms. Trace:
I20260320 15:27:43.054122 29643 log.cc:1199] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:43.054306 29640 rpcz_store.cc:276] 0320 15:27:34.073037 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.073093 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:42.955988 (+8882895us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:42.955997 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.022140 (+ 66143us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.052500 (+ 30360us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.054190 (+ 1690us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2193,"tablet-delete.queue_time_us":8881192}
I20260320 15:27:43.054682 29640 tablet_replica.cc:333] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.054912 29640 raft_consensus.cc:2243] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.054857 28736 ts_tablet_manager.cc:1434] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.041s user 0.004s sys 0.000s
I20260320 15:27:43.055269 29640 pending_rounds.cc:70] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:43.055327 29640 pending_rounds.cc:77] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387801544527872 op_type: NO_OP noop_request { }
I20260320 15:27:43.055428 29640 raft_consensus.cc:2889] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:43.057637 28736 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:43.057708 29643 ts_tablet_manager.cc:1950] T 43a0fadc7dac4e21ad034e4cd2d70826 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.064407 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet e7bd0616f5604f8fbb8edd8e9c230f39 (table default.Table_17c9ae5b739344f4ab272260023b6a74 [id=30b8ab71f82a42ebbb2976696e1ecb4e]) successfully deleted
I20260320 15:27:43.055522 29640 raft_consensus.cc:2272] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.064847 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7be03035dc3d44f9aa5b43b49281a940 (table default.Table_e9195525674d41a89bd4e262baa86a81 [id=34e8e54dc6ea430aa5425ac1e8aea9ac]) successfully deleted
I20260320 15:27:43.065322 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:43.065691 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]). Expected version 1 got 0
I20260320 15:27:43.066095 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:43.071482 29640 ts_tablet_manager.cc:1916] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.071494 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 146) took 8846 ms. Trace:
I20260320 15:27:43.071738 29643 rpcz_store.cc:276] 0320 15:27:34.224900 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.224942 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:42.955125 (+8730183us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:42.955180 (+ 55us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.004168 (+ 48988us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.052295 (+ 48127us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.071435 (+ 19140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5812,"tablet-delete.queue_time_us":8730025}
I20260320 15:27:43.072886 29643 tablet_replica.cc:333] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:43.075165 28524 consensus_peers.cc:597] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bdd8f5ccbd35418684fd27459c7a6b9f. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.084960 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 43a0fadc7dac4e21ad034e4cd2d70826 (table default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]) successfully deleted
I20260320 15:27:43.085337 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:43.086165 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:43.090811 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:43.093730 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:43.073319 29643 raft_consensus.cc:2243] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.098428 29678 ts_tablet_manager.cc:1929] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.098882 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]). Expected version 2 got 0
I20260320 15:27:43.099130 29678 log.cc:1199] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/796c907919df47bf9369e45a1248f96b
I20260320 15:27:43.092350 28734 tablet_bootstrap.cc:654] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.098868 29643 raft_consensus.cc:2272] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.100006 29678 ts_tablet_manager.cc:1950] T 796c907919df47bf9369e45a1248f96b P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.100212 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 80810790e42947a69ce6140feb9f7cbc (table default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784]). Expected version 2 got 0
W20260320 15:27:43.103469 28524 consensus_peers.cc:597] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ed29146bff74e86a366d3c6329176f4. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.103632 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ba4eb20eb0f1403282030727757031f7 (table default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0]). Expected version 1 got 0
I20260320 15:27:43.104565 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:43.104270 29643 ts_tablet_manager.cc:1916] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.105813 29640 ts_tablet_manager.cc:1929] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.106240 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:43.106660 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:43.107188 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:43.109061 30344 raft_consensus.cc:493] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.109232 30344 raft_consensus.cc:515] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.110327 29640 log.cc:1199] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ffe278739bbc4ddab848af9c25241380
I20260320 15:27:43.110767 29640 ts_tablet_manager.cc:1950] T ffe278739bbc4ddab848af9c25241380 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.111120 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 133) took 9037 ms. Trace:
I20260320 15:27:43.110687 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.111203 29640 rpcz_store.cc:276] 0320 15:27:34.073390 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.073446 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:43.054633 (+8981187us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:43.054642 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.091187 (+ 36545us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.105688 (+ 14501us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.111101 (+ 5413us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8979822}
I20260320 15:27:43.111591 29640 tablet_replica.cc:333] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.111809 29640 raft_consensus.cc:2243] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.112159 29640 raft_consensus.cc:2272] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:43.112125 28382 leader_election.cc:343] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.112808 29640 ts_tablet_manager.cc:1916] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.111821 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ffe278739bbc4ddab848af9c25241380 (table default.Table_6cadeb80b9df43149080a924bb38ae8e [id=275f662fcfc14843bbc5fc3a577d6d22]) successfully deleted
I20260320 15:27:43.113729 28733 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.114014 28733 ts_tablet_manager.cc:1403] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.079s user 0.004s sys 0.000s
I20260320 15:27:43.114074 28904 catalog_manager.cc:5671] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:43.114583 30344 leader_election.cc:290] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.114745 28904 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:43.114710 30155 consensus_queue.cc:1048] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:43.115818 28890 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71db28879e554f578b8a7b0e14419ab7 has no consensus state
I20260320 15:27:43.116006 28890 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 111) took 2635 ms. Trace:
I20260320 15:27:43.116083 28890 rpcz_store.cc:276] 0320 15:27:40.480812 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.541156 (+2060344us) service_pool.cc:225] Handling call
0320 15:27:42.541664 (+ 508us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:42.542008 (+ 344us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:42.542224 (+ 216us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:42.542360 (+ 136us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:42.562203 (+ 19843us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:43.115954 (+553751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.542903 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.543025 (+ 122us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.543031 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:42.543033 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:42.543135 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.543148 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.543152 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:42.543223 (+ 71us) tablet.cc:700] Row locks acquired
0320 15:27:42.543226 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:42.543261 (+ 35us) write_op.cc:270] Start()
0320 15:27:42.543319 (+ 58us) write_op.cc:276] Timestamp: P: 1774020462543210 usec, L: 0
0320 15:27:42.543322 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.543506 (+ 184us) log.cc:844] Serialized 405 byte log entry
0320 15:27:42.547764 (+ 4258us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.547932 (+ 168us) write_op.cc:301] APPLY: starting
0320 15:27:42.547961 (+ 29us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.548018 (+ 57us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.548058 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.548132 (+ 74us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.548182 (+ 50us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:42.548194 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:42.548315 (+ 121us) log.cc:844] Serialized 23 byte log entry
0320 15:27:42.548392 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.548414 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:42.548420 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:42.548500 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.548504 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":569,"apply.run_wall_time_us":569,"num_ops":1,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":731,"prepare.run_wall_time_us":732,"raft.queue_time_us":28,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":252,"replication_time_us":4405,"spinlock_wait_cycles":3456}]]}
I20260320 15:27:43.117501 28902 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:43.118119 28902 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 96) took 4375 ms. Trace:
I20260320 15:27:43.118227 28902 rpcz_store.cc:276] 0320 15:27:38.742920 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.506213 (+2763293us) service_pool.cc:225] Handling call
0320 15:27:41.585242 (+ 79029us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:41.585452 (+ 210us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:41.585533 (+ 81us) catalog_manager.cc:2824] Locking tablets
0320 15:27:41.798250 (+212717us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:41.809740 (+ 11490us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:41.957544 (+147804us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:43.117427 (+1159883us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:43.118047 (+ 620us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:41.800039 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:41.800171 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:41.800179 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:41.800181 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:41.800340 (+ 159us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:41.800353 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:41.800359 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:41.800491 (+ 132us) tablet.cc:700] Row locks acquired
0320 15:27:41.800494 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:41.800537 (+ 43us) write_op.cc:270] Start()
0320 15:27:41.800576 (+ 39us) write_op.cc:276] Timestamp: P: 1774020461800513 usec, L: 0
0320 15:27:41.800579 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:41.800833 (+ 254us) log.cc:844] Serialized 725 byte log entry
0320 15:27:41.804669 (+ 3836us) op_driver.cc:464] REPLICATION: finished
0320 15:27:41.809040 (+ 4371us) write_op.cc:301] APPLY: starting
0320 15:27:41.809060 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:41.809115 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:41.809148 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:41.809240 (+ 92us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:41.809282 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:41.809291 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:41.809388 (+ 97us) log.cc:844] Serialized 29 byte log entry
0320 15:27:41.809484 (+ 96us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.809504 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:41.809511 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:41.809606 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.809613 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4306,"apply.run_cpu_time_us":553,"apply.run_wall_time_us":553,"num_ops":2,"prepare.queue_time_us":982,"prepare.run_cpu_time_us":920,"prepare.run_wall_time_us":921,"raft.queue_time_us":45,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":3564,"replication_time_us":4052}]]}
I20260320 15:27:43.124745 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:41 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.125190 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:41 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:43.119343 28733 raft_consensus.cc:359] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.143277 28733 raft_consensus.cc:385] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.150400 29939 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:43.147945 28733 raft_consensus.cc:740] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.150671 29939 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:43.152261 29939 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:43.152390 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.158871 30344 raft_consensus.cc:493] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.154019 28733 consensus_queue.cc:260] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.159139 30344 raft_consensus.cc:515] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.161005 30344 leader_election.cc:290] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.168143 30344 raft_consensus.cc:493] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.168329 30344 raft_consensus.cc:515] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.169014 30344 leader_election.cc:290] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.170006 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:43.170719 28382 leader_election.cc:343] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.174772 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:43.175806 28382 leader_election.cc:343] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.181874 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.181970 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 151) took 8898 ms. Trace:
I20260320 15:27:43.182075 29678 rpcz_store.cc:276] 0320 15:27:34.283091 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.283186 (+ 95us) service_pool.cc:225] Handling call
0320 15:27:43.027315 (+8744129us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:43.027326 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.080377 (+ 53051us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.098290 (+ 17913us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.181950 (+ 83660us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8740622}
I20260320 15:27:43.188110 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:41 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.188656 29678 tablet_replica.cc:333] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.188871 29678 raft_consensus.cc:2243] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Raft consensus shutting down.
W20260320 15:27:43.189713 28383 leader_election.cc:343] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.189783 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 796c907919df47bf9369e45a1248f96b (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]) successfully deleted
I20260320 15:27:43.189869 28383 leader_election.cc:304] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.190157 30407 raft_consensus.cc:493] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80636bb608a1431b955576e296ff63bb)
I20260320 15:27:43.190285 30344 raft_consensus.cc:2749] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.190341 30407 raft_consensus.cc:515] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.190495 28734 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.190809 28734 ts_tablet_manager.cc:1403] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.138s user 0.007s sys 0.002s
I20260320 15:27:43.193527 29678 raft_consensus.cc:2272] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.194186 30407 leader_election.cc:290] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:43.194280 28736 tablet_bootstrap.cc:654] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.194931 29678 ts_tablet_manager.cc:1916] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:43.197365 28240 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:43.207595 28734 raft_consensus.cc:359] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.219599 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.220057 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.221343 28733 ts_tablet_manager.cc:1434] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.107s user 0.005s sys 0.000s
I20260320 15:27:43.221541 28734 raft_consensus.cc:385] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.221719 28734 raft_consensus.cc:740] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.222021 28734 consensus_queue.cc:260] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.222457 29678 ts_tablet_manager.cc:1929] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:43.222597 29678 log.cc:1199] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:43.223094 29678 ts_tablet_manager.cc:1950] T 5438fdc6e6944e79ac8aecd0ef774467 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.223443 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 152) took 8937 ms. Trace:
I20260320 15:27:43.223559 29678 rpcz_store.cc:276] 0320 15:27:34.285762 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.285907 (+ 145us) service_pool.cc:225] Handling call
0320 15:27:43.186338 (+8900431us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:43.188357 (+ 2019us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.217028 (+ 28671us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.222160 (+ 5132us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.223426 (+ 1266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8898208}
I20260320 15:27:43.223973 29678 tablet_replica.cc:333] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.224231 29678 raft_consensus.cc:2243] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.224432 29678 raft_consensus.cc:2272] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.225135 29678 ts_tablet_manager.cc:1916] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:43.228340 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6bbc5d5f49a04216a352ddb1b571dce0 consensus state has no leader
I20260320 15:27:43.238935 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 71db28879e554f578b8a7b0e14419ab7 (DEFAULT_TABLE table=default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.240011 28734 ts_tablet_manager.cc:1434] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.049s user 0.001s sys 0.001s
I20260320 15:27:43.240372 28734 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.241719 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 71db28879e554f578b8a7b0e14419ab7 (DEFAULT_TABLE table=default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.242182 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71db28879e554f578b8a7b0e14419ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.248306 30344 raft_consensus.cc:493] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.248471 30344 raft_consensus.cc:515] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.249024 29640 ts_tablet_manager.cc:1929] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.249189 29640 log.cc:1199] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:43.249625 29640 ts_tablet_manager.cc:1950] T 5fa5f6fe801b4452bf294a9ae23bb163 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.254316 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 366df9b29cee4b2b9dfd72b0061f175a (DEFAULT_TABLE table=default.Table_7abb6e0bedbe439daf8e7bbede311b1a [id=73b84a8ea2d14d62ab611990602bcec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.254405 28736 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:43.254976 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:43.255409 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 366df9b29cee4b2b9dfd72b0061f175a (DEFAULT_TABLE table=default.Table_7abb6e0bedbe439daf8e7bbede311b1a [id=73b84a8ea2d14d62ab611990602bcec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.255962 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:43.256037 28736 ts_tablet_manager.cc:1403] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.199s user 0.012s sys 0.002s
W20260320 15:27:43.256592 28383 leader_election.cc:343] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.256739 28383 leader_election.cc:304] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:43.256870 28524 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: aafd1e123cd043e7828140d2443e047f
I20260320 15:27:43.256875 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:43.257009 28524 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.257196 30313 raft_consensus.cc:2749] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:43.257586 28383 leader_election.cc:343] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.257683 28383 leader_election.cc:304] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.257762 30155 raft_consensus.cc:2749] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:43.257922 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:43.258019 30313 raft_consensus.cc:2749] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.258466 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5438fdc6e6944e79ac8aecd0ef774467 (table default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]) successfully deleted
I20260320 15:27:43.258451 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:43.258631 30257 raft_consensus.cc:493] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.258783 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:43.258770 30257 raft_consensus.cc:515] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.258790 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.258929 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:43.259375 30257 leader_election.cc:290] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:43.259732 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
W20260320 15:27:43.260028 28239 leader_election.cc:343] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:43.260133 28239 leader_election.cc:304] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.260200 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fa54115d4df74bec85970057b5c3dd51 (table default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc]). Expected version 1 got 0
W20260320 15:27:43.260236 28382 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.260461 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:43.260936 30155 consensus_queue.cc:1048] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.261422 30257 raft_consensus.cc:2749] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:43.261442 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:43.261998 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
W20260320 15:27:43.262081 28240 leader_election.cc:343] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c
I20260320 15:27:43.261737 28736 raft_consensus.cc:359] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.262533 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fa691a26114657ad768545a30dfd3c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.262702 28574 raft_consensus.cc:2468] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:43.262714 28736 raft_consensus.cc:385] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.263388 28736 raft_consensus.cc:740] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.263803 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae72bb24d5846859c6ed8722802a642 (DEFAULT_TABLE table=default.Table_c30a162e7c4c49db8ef902e38586b19a [id=b21ba6332cf64bf1a7ccb67141b7ec4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.265858 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:43.266518 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
W20260320 15:27:43.267920 28524 consensus_peers.cc:597] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa5618796f1a4ff3ab4c1209dad0aede. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.267001 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:43.269340 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:43.269838 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:43.270829 28736 consensus_queue.cc:260] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.271847 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae72bb24d5846859c6ed8722802a642 (DEFAULT_TABLE table=default.Table_c30a162e7c4c49db8ef902e38586b19a [id=b21ba6332cf64bf1a7ccb67141b7ec4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.271982 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:43.272579 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 71db28879e554f578b8a7b0e14419ab7 (DEFAULT_TABLE table=default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.273018 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:43.273094 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71db28879e554f578b8a7b0e14419ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.273334 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ce8595157679444b959b569dbb7d8df0 (table default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191]). Expected version 1 got 0
I20260320 15:27:43.273865 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:43.273662 28736 ts_tablet_manager.cc:1434] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:43.274259 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:43.274537 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:43.274770 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 2f5beaa1bc1e4465be34c8e86f131559 (table default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc]). Expected version 2 got 0
I20260320 15:27:43.274941 28733 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.275116 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:43.275252 28736 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:43.275558 28890 catalog_manager.cc:5671] T e0e7ae1ff302428abf6cc592bbca0aa9 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:43.276674 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 (DEFAULT_TABLE table=default.Table_dfac030930b54821b08b28ee38374406 [id=1e44e6a2a1b1477a92784ae9a36d52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.276980 28733 tablet_bootstrap.cc:654] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.277007 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:43.277613 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:43.278235 28890 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:43.285090 30344 leader_election.cc:290] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.286068 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (DEFAULT_TABLE table=default.Table_f832b79a92ff48718c372c5aee5ce446 [id=035b44fe37d94257babdcd3ffe38d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.286660 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (DEFAULT_TABLE table=default.Table_f832b79a92ff48718c372c5aee5ce446 [id=035b44fe37d94257babdcd3ffe38d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.287509 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366df9b29cee4b2b9dfd72b0061f175a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.289268 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 366df9b29cee4b2b9dfd72b0061f175a (DEFAULT_TABLE table=default.Table_7abb6e0bedbe439daf8e7bbede311b1a [id=73b84a8ea2d14d62ab611990602bcec5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.289883 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.291272 29678 ts_tablet_manager.cc:1929] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.291364 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae72bb24d5846859c6ed8722802a642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.291384 29678 log.cc:1199] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/442ca186c9d4449dade39271dec5c608
I20260320 15:27:43.292906 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 (DEFAULT_TABLE table=default.Table_dfac030930b54821b08b28ee38374406 [id=1e44e6a2a1b1477a92784ae9a36d52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.293298 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.294657 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71db28879e554f578b8a7b0e14419ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.295502 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366df9b29cee4b2b9dfd72b0061f175a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.298095 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 5ae72bb24d5846859c6ed8722802a642 (DEFAULT_TABLE table=default.Table_c30a162e7c4c49db8ef902e38586b19a [id=b21ba6332cf64bf1a7ccb67141b7ec4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.298754 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 (DEFAULT_TABLE table=default.Table_dfac030930b54821b08b28ee38374406 [id=1e44e6a2a1b1477a92784ae9a36d52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.299384 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (DEFAULT_TABLE table=default.Table_f832b79a92ff48718c372c5aee5ce446 [id=035b44fe37d94257babdcd3ffe38d068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.299734 28239 leader_election.cc:304] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.299810 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.299976 30257 raft_consensus.cc:2804] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.300064 30257 raft_consensus.cc:493] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.300135 30257 raft_consensus.cc:3060] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.302708 30257 raft_consensus.cc:515] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.303388 30257 leader_election.cc:290] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:43.304308 28240 leader_election.cc:343] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c
I20260320 15:27:43.306335 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae72bb24d5846859c6ed8722802a642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.307775 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.310039 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.316066 28733 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.316322 28733 ts_tablet_manager.cc:1403] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.000s
I20260320 15:27:43.317426 28733 raft_consensus.cc:359] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.317771 28733 raft_consensus.cc:385] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.317888 28733 raft_consensus.cc:740] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.318267 28733 consensus_queue.cc:260] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.319034 28733 ts_tablet_manager.cc:1434] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:43.321820 28733 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.323921 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ae72bb24d5846859c6ed8722802a642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.325052 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 366df9b29cee4b2b9dfd72b0061f175a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.327883 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.330452 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 128) took 9258 ms. Trace:
I20260320 15:27:43.330560 29640 rpcz_store.cc:276] 0320 15:27:34.071987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.072031 (+ 44us) service_pool.cc:225] Handling call
0320 15:27:43.111520 (+9039489us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:43.111551 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.185184 (+ 73633us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.248873 (+ 63689us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.330424 (+ 81551us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9035597}
I20260320 15:27:43.330914 29640 tablet_replica.cc:333] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.331091 29640 raft_consensus.cc:2243] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.331120 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 5fa5f6fe801b4452bf294a9ae23bb163 (table default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]) successfully deleted
I20260320 15:27:43.331447 29640 raft_consensus.cc:2272] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.331756 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.332083 29640 ts_tablet_manager.cc:1916] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.339147 29640 ts_tablet_manager.cc:1929] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.339303 29640 log.cc:1199] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/533105ad6e054fe3a4db32855bc688bd
I20260320 15:27:43.341661 29640 ts_tablet_manager.cc:1950] T 533105ad6e054fe3a4db32855bc688bd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.342550 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 135) took 9261 ms. Trace:
I20260320 15:27:43.342660 29640 rpcz_store.cc:276] 0320 15:27:34.080864 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.080914 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:43.330863 (+9249949us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:43.330875 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.335363 (+ 4488us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.339026 (+ 3663us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.342527 (+ 3501us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9235289}
I20260320 15:27:43.343086 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 134) took 9267 ms. Trace:
I20260320 15:27:43.343132 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 533105ad6e054fe3a4db32855bc688bd (table default.Table_0a6897bb762e4cd6af9d946b1f0cbe9d [id=7455037efa3e4bd5bed993f6d0d05b4f]) successfully deleted
I20260320 15:27:43.343168 29640 rpcz_store.cc:276] 0320 15:27:34.075822 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.076043 (+ 221us) service_pool.cc:225] Handling call
0320 15:27:43.342954 (+9266911us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:43.342964 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.343068 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9247253}
I20260320 15:27:43.343441 29640 tablet_replica.cc:333] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:43.343570 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41da3cb06b1147fbae2695bc5c9ffaa5 already in progress: creating tablet
I20260320 15:27:43.343688 29640 raft_consensus.cc:2243] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.343861 29640 raft_consensus.cc:2272] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.344558 29640 ts_tablet_manager.cc:1916] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:43.344767 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 45065.8us
W20260320 15:27:43.345890 28383 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.346019 28383 leader_election.cc:304] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.346460 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fa691a26114657ad768545a30dfd3c" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.346632 28574 raft_consensus.cc:3060] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.347076 29678 ts_tablet_manager.cc:1950] T 442ca186c9d4449dade39271dec5c608 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.347496 28736 tablet_bootstrap.cc:654] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.348398 30155 consensus_queue.cc:1048] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.349612 28574 raft_consensus.cc:2468] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:43.353513 30344 raft_consensus.cc:2749] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.355136 28278 tablet_service.cc:1511] Processing CreateTablet for tablet ddf7f083f2774d76a149afbd9bca9f05 (DEFAULT_TABLE table=default.Table_e2cf21f098cf48c494849f6e8d339dc5 [id=f0baf377b4674a659b86b1dfd081e36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.355679 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf7f083f2774d76a149afbd9bca9f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.360569 29640 ts_tablet_manager.cc:1929] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.361199 29640 log.cc:1199] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:43.361637 29640 ts_tablet_manager.cc:1950] T 2ee4a0d559ce4dfea878bd3f1c03c09a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.362366 28239 leader_election.cc:304] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.362624 30256 raft_consensus.cc:2804] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.362739 30256 raft_consensus.cc:697] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:43.362978 30256 consensus_queue.cc:237] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:43.364355 30421 log.cc:927] Time spent T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260320 15:27:43.368795 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 136) took 9269 ms. Trace:
I20260320 15:27:43.369830 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 9020f4aaf3024802a83bc1f9b557a86c (DEFAULT_TABLE table=default.Table_db4a9c34ad40421aa3f8f70873479784 [id=0185a7d1388547e78fa406f2a0be52c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.370275 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9020f4aaf3024802a83bc1f9b557a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.371110 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 9020f4aaf3024802a83bc1f9b557a86c (DEFAULT_TABLE table=default.Table_db4a9c34ad40421aa3f8f70873479784 [id=0185a7d1388547e78fa406f2a0be52c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.371567 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9020f4aaf3024802a83bc1f9b557a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.371771 28734 tablet_bootstrap.cc:654] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.371946 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 153) took 9085 ms. Trace:
I20260320 15:27:43.372196 29643 ts_tablet_manager.cc:1929] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.372053 29678 rpcz_store.cc:276] 0320 15:27:34.286715 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.286817 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:43.223916 (+8937099us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:43.223928 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.260653 (+ 36725us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.291164 (+ 30511us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.371918 (+ 80754us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8928739}
I20260320 15:27:43.372318 29643 log.cc:1199] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:43.372469 29678 tablet_replica.cc:333] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.372685 29678 raft_consensus.cc:2243] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.372718 29643 ts_tablet_manager.cc:1950] T e7bd0616f5604f8fbb8edd8e9c230f39 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.372869 29678 raft_consensus.cc:2272] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.373122 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ddf7f083f2774d76a149afbd9bca9f05 (DEFAULT_TABLE table=default.Table_e2cf21f098cf48c494849f6e8d339dc5 [id=f0baf377b4674a659b86b1dfd081e36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.373500 29678 ts_tablet_manager.cc:1916] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.373535 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf7f083f2774d76a149afbd9bca9f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.368952 29640 rpcz_store.cc:276] 0320 15:27:34.098880 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.098936 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:43.343401 (+9244465us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:43.343410 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.357223 (+ 13813us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.360468 (+ 3245us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.368759 (+ 8291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9223473}
I20260320 15:27:43.375934 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 2ee4a0d559ce4dfea878bd3f1c03c09a (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]) successfully deleted
I20260320 15:27:43.379395 29640 tablet_replica.cc:333] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.380251 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 442ca186c9d4449dade39271dec5c608 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
I20260320 15:27:43.380366 29640 raft_consensus.cc:2243] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.380698 29640 pending_rounds.cc:70] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
I20260320 15:27:43.380770 29640 pending_rounds.cc:77] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387810577629184 op_type: NO_OP noop_request { }
I20260320 15:27:43.380887 29640 raft_consensus.cc:2889] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:43.380985 29640 raft_consensus.cc:2272] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.381651 29640 ts_tablet_manager.cc:1916] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.381847 28552 tablet_service.cc:1511] Processing CreateTablet for tablet d823ab41194542fba5540e53e9f4680b (DEFAULT_TABLE table=default.Table_799abb7057d3405db01b42c86431f01d [id=2a857f7bb9bb4f6b983db9cfbad54c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.384322 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 9020f4aaf3024802a83bc1f9b557a86c (DEFAULT_TABLE table=default.Table_db4a9c34ad40421aa3f8f70873479784 [id=0185a7d1388547e78fa406f2a0be52c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.385807 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9020f4aaf3024802a83bc1f9b557a86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.387890 30344 raft_consensus.cc:493] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.389078 28733 tablet_bootstrap.cc:654] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.390477 28425 tablet_service.cc:1511] Processing CreateTablet for tablet d823ab41194542fba5540e53e9f4680b (DEFAULT_TABLE table=default.Table_799abb7057d3405db01b42c86431f01d [id=2a857f7bb9bb4f6b983db9cfbad54c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.390596 28409 tablet_service.cc:1511] Processing CreateTablet for tablet ddf7f083f2774d76a149afbd9bca9f05 (DEFAULT_TABLE table=default.Table_e2cf21f098cf48c494849f6e8d339dc5 [id=f0baf377b4674a659b86b1dfd081e36e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.390898 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d823ab41194542fba5540e53e9f4680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.384042 28278 tablet_service.cc:1511] Processing CreateTablet for tablet d823ab41194542fba5540e53e9f4680b (DEFAULT_TABLE table=default.Table_799abb7057d3405db01b42c86431f01d [id=2a857f7bb9bb4f6b983db9cfbad54c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.391584 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d823ab41194542fba5540e53e9f4680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.388092 30344 raft_consensus.cc:515] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.392908 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 147) took 9167 ms. Trace:
I20260320 15:27:43.393023 29643 rpcz_store.cc:276] 0320 15:27:34.225233 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.225278 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:43.072758 (+8847480us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:43.072806 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.216391 (+143585us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.372076 (+155685us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.392883 (+ 20807us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8847323}
I20260320 15:27:43.393426 29643 tablet_replica.cc:333] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.393661 29643 raft_consensus.cc:2243] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.393859 29643 raft_consensus.cc:2272] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.394491 29643 ts_tablet_manager.cc:1916] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.395108 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet e7bd0616f5604f8fbb8edd8e9c230f39 (table default.Table_17c9ae5b739344f4ab272260023b6a74 [id=30b8ab71f82a42ebbb2976696e1ecb4e]) successfully deleted
I20260320 15:27:43.395182 28736 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:43.396023 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d823ab41194542fba5540e53e9f4680b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.396085 28736 ts_tablet_manager.cc:1403] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.121s user 0.004s sys 0.002s
I20260320 15:27:43.402269 30256 raft_consensus.cc:493] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.402483 30256 raft_consensus.cc:515] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.403311 30256 leader_election.cc:290] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:43.404369 28239 leader_election.cc:343] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
I20260320 15:27:43.404989 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 609ead6771b34a9ab21b5824c33bec2f (DEFAULT_TABLE table=default.Table_263dcedacaae47cca5681f61731b53d0 [id=7dd6ef783235481684c83c540f45ac5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.405390 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609ead6771b34a9ab21b5824c33bec2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.407951 30344 leader_election.cc:290] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.408088 30344 raft_consensus.cc:493] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.408229 30344 raft_consensus.cc:515] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.408880 30344 leader_election.cc:290] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:43.409559 28240 leader_election.cc:343] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
I20260320 15:27:43.409684 28240 leader_election.cc:304] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.413774 30256 raft_consensus.cc:2749] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.416612 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 609ead6771b34a9ab21b5824c33bec2f (DEFAULT_TABLE table=default.Table_263dcedacaae47cca5681f61731b53d0 [id=7dd6ef783235481684c83c540f45ac5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.417289 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609ead6771b34a9ab21b5824c33bec2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.418768 30155 raft_consensus.cc:493] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.418970 30155 raft_consensus.cc:515] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.419857 30155 leader_election.cc:290] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.421612 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.421861 28446 raft_consensus.cc:2468] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.423146 28524 leader_election.cc:304] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.423638 30155 raft_consensus.cc:2804] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.423724 30155 raft_consensus.cc:493] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.423792 30155 raft_consensus.cc:3060] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.426893 28733 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.427042 28733 ts_tablet_manager.cc:1403] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.105s user 0.006s sys 0.010s
I20260320 15:27:43.427142 30155 raft_consensus.cc:515] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.427898 28733 raft_consensus.cc:359] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.428066 28733 raft_consensus.cc:385] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.428110 28733 raft_consensus.cc:740] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.428351 28733 consensus_queue.cc:260] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.428951 28733 ts_tablet_manager.cc:1434] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:43.429648 28733 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.430408 30313 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:43.430557 30313 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:43.431200 30313 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.433324 30155 leader_election.cc:290] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.434224 28733 tablet_bootstrap.cc:654] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.438915 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 01828bedfdd94d62b780a9d5a40b17a5 (DEFAULT_TABLE table=default.Table_d71bcddc2804450dbb050fdc6c54338a [id=e692c850a8dc4913b96e99ec537ae2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.446064 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 01828bedfdd94d62b780a9d5a40b17a5 (DEFAULT_TABLE table=default.Table_d71bcddc2804450dbb050fdc6c54338a [id=e692c850a8dc4913b96e99ec537ae2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.446561 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01828bedfdd94d62b780a9d5a40b17a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.448688 28733 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.449100 28733 ts_tablet_manager.cc:1403] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.007s
I20260320 15:27:43.451508 28733 raft_consensus.cc:359] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.452239 28733 raft_consensus.cc:385] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.452391 28733 raft_consensus.cc:740] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.453083 28733 consensus_queue.cc:260] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.454459 28733 ts_tablet_manager.cc:1434] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260320 15:27:43.455791 28733 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.460001 28733 tablet_bootstrap.cc:654] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.461273 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.461549 28446 raft_consensus.cc:3060] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.463924 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddf7f083f2774d76a149afbd9bca9f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.466605 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01828bedfdd94d62b780a9d5a40b17a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.467379 30395 raft_consensus.cc:493] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.467710 30395 raft_consensus.cc:515] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.468645 30395 leader_election.cc:290] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.472481 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.472720 28445 raft_consensus.cc:2468] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.473106 28733 tablet_bootstrap.cc:492] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.473425 28733 ts_tablet_manager.cc:1403] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.007s
I20260320 15:27:43.473963 28524 leader_election.cc:304] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.474545 30394 raft_consensus.cc:2804] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.474622 30394 raft_consensus.cc:493] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.474680 30394 raft_consensus.cc:3060] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.475302 28733 raft_consensus.cc:359] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.475656 28733 raft_consensus.cc:385] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.475768 28733 raft_consensus.cc:740] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.476181 28733 consensus_queue.cc:260] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.477869 28733 ts_tablet_manager.cc:1434] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:43.478386 30394 raft_consensus.cc:515] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.478535 28733 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.479094 30394 leader_election.cc:290] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.480685 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.480916 28441 raft_consensus.cc:3060] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.482849 28733 tablet_bootstrap.cc:654] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.486591 28446 raft_consensus.cc:2468] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.487830 28555 tablet_service.cc:1511] Processing CreateTablet for tablet a84e8b2025a14984b69b4fb2230b9263 (DEFAULT_TABLE table=default.Table_c24e53f814fb49259ccd5a2d2054eeb3 [id=29afe6b4b6eb426888494923d61a379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.488256 28524 leader_election.cc:304] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.488566 30395 raft_consensus.cc:2804] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.488687 30395 raft_consensus.cc:697] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.489022 28441 raft_consensus.cc:2468] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.488973 30395 consensus_queue.cc:237] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.489598 28524 leader_election.cc:304] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.489894 29939 raft_consensus.cc:493] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.490034 29939 raft_consensus.cc:515] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.490221 30155 raft_consensus.cc:2804] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.490489 30155 raft_consensus.cc:697] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.490731 29939 leader_election.cc:290] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.490810 30155 consensus_queue.cc:237] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.491194 29939 raft_consensus.cc:493] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.491365 29939 raft_consensus.cc:515] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.492014 29939 leader_election.cc:290] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.495373 28733 tablet_bootstrap.cc:492] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.495702 28733 ts_tablet_manager.cc:1403] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.003s
I20260320 15:27:43.497563 28733 raft_consensus.cc:359] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.497902 28733 raft_consensus.cc:385] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.498026 28733 raft_consensus.cc:740] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.498510 28733 consensus_queue.cc:260] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.499737 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.499737 28733 ts_tablet_manager.cc:1434] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:43.500018 28441 raft_consensus.cc:2468] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.500481 28524 leader_election.cc:304] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.500813 30394 raft_consensus.cc:2804] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.500905 30394 raft_consensus.cc:493] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.500981 30394 raft_consensus.cc:3060] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.500929 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.501101 28446 raft_consensus.cc:2468] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.501627 28524 leader_election.cc:304] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.502139 30359 raft_consensus.cc:2804] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.502221 30359 raft_consensus.cc:493] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.502298 30359 raft_consensus.cc:3060] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.504571 28424 tablet_service.cc:1511] Processing CreateTablet for tablet a84e8b2025a14984b69b4fb2230b9263 (DEFAULT_TABLE table=default.Table_c24e53f814fb49259ccd5a2d2054eeb3 [id=29afe6b4b6eb426888494923d61a379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.505046 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84e8b2025a14984b69b4fb2230b9263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.505174 30359 raft_consensus.cc:515] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.505856 30359 leader_election.cc:290] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.507211 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.507404 28446 raft_consensus.cc:3060] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.510571 28446 raft_consensus.cc:2468] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.511372 28524 leader_election.cc:304] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.512574 30355 raft_consensus.cc:2804] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.518885 29643 ts_tablet_manager.cc:1929] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.519044 29643 log.cc:1199] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:43.519431 29643 ts_tablet_manager.cc:1950] T 3ed29146bff74e86a366d3c6329176f4 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.521250 28736 raft_consensus.cc:359] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.521466 28736 raft_consensus.cc:385] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.521539 28736 raft_consensus.cc:740] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.521746 28736 consensus_queue.cc:260] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.521850 30394 raft_consensus.cc:515] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.522909 30394 leader_election.cc:290] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.523428 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.523697 28446 raft_consensus.cc:3060] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.530237 28446 raft_consensus.cc:2468] T a3bd9c79d2d74e878647f97e66a9bfca P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.526755 29640 ts_tablet_manager.cc:1929] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.530968 29640 log.cc:1199] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/fc009401bb3d440f9d36e080efef1204
I20260320 15:27:43.531816 28736 ts_tablet_manager.cc:1434] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.136s user 0.000s sys 0.002s
I20260320 15:27:43.531759 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.532143 28736 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:43.532308 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.533043 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 609ead6771b34a9ab21b5824c33bec2f (DEFAULT_TABLE table=default.Table_263dcedacaae47cca5681f61731b53d0 [id=7dd6ef783235481684c83c540f45ac5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.533126 29640 ts_tablet_manager.cc:1950] T fc009401bb3d440f9d36e080efef1204 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.530754 28524 leader_election.cc:304] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.535676 30394 raft_consensus.cc:2804] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.536011 30394 raft_consensus.cc:697] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.536362 30394 consensus_queue.cc:237] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.537142 28734 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.537305 28734 ts_tablet_manager.cc:1403] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.297s user 0.011s sys 0.003s
W20260320 15:27:43.539686 28383 leader_election.cc:343] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.539916 28383 leader_election.cc:343] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.540066 28383 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: aafd1e123cd043e7828140d2443e047f
I20260320 15:27:43.540294 28277 tablet_service.cc:1511] Processing CreateTablet for tablet a84e8b2025a14984b69b4fb2230b9263 (DEFAULT_TABLE table=default.Table_c24e53f814fb49259ccd5a2d2054eeb3 [id=29afe6b4b6eb426888494923d61a379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.540769 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84e8b2025a14984b69b4fb2230b9263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.541491 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 01828bedfdd94d62b780a9d5a40b17a5 (DEFAULT_TABLE table=default.Table_d71bcddc2804450dbb050fdc6c54338a [id=e692c850a8dc4913b96e99ec537ae2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:43.541605 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 148) took 9257 ms. Trace:
I20260320 15:27:43.541847 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01828bedfdd94d62b780a9d5a40b17a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.541694 29643 rpcz_store.cc:276] 0320 15:27:34.285172 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.285382 (+ 210us) service_pool.cc:225] Handling call
0320 15:27:43.393378 (+9107996us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:43.393390 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.444353 (+ 50963us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.518752 (+ 74399us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.541582 (+ 22830us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9107771}
I20260320 15:27:43.542045 29643 tablet_replica.cc:333] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.542254 29643 raft_consensus.cc:2243] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.542426 29643 raft_consensus.cc:2272] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.543174 29643 ts_tablet_manager.cc:1916] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.543709 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.544288 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84e8b2025a14984b69b4fb2230b9263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.544420 28733 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.546453 28733 tablet_bootstrap.cc:654] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.548205 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 3ed29146bff74e86a366d3c6329176f4 (table default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]) successfully deleted
I20260320 15:27:43.548589 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:43.548723 30424 log.cc:927] Time spent T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Append to log took a long time: real 0.153s user 0.000s sys 0.000s
I20260320 15:27:43.548797 28575 raft_consensus.cc:2468] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:43.548990 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 609ead6771b34a9ab21b5824c33bec2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:43.549065 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.549788 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.549948 28576 raft_consensus.cc:2468] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:43.550499 29678 ts_tablet_manager.cc:1929] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.550604 29678 log.cc:1199] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7f0242a7968541f480367b45cda5d9cc
I20260320 15:27:43.550964 29678 ts_tablet_manager.cc:1950] T 7f0242a7968541f480367b45cda5d9cc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.551712 28382 leader_election.cc:304] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.551797 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 154) took 9263 ms. Trace:
I20260320 15:27:43.551887 29678 rpcz_store.cc:276] 0320 15:27:34.288606 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.288699 (+ 93us) service_pool.cc:225] Handling call
0320 15:27:43.372419 (+9083720us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:43.372430 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.524616 (+152186us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.550386 (+ 25770us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.551776 (+ 1390us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9077098}
I20260320 15:27:43.552111 30313 raft_consensus.cc:2804] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.552187 30313 raft_consensus.cc:493] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.552213 29678 tablet_replica.cc:333] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.552245 30313 raft_consensus.cc:3060] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.552413 29678 raft_consensus.cc:2243] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.552574 29678 raft_consensus.cc:2272] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.553129 29678 ts_tablet_manager.cc:1916] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.554566 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 138) took 9439 ms. Trace:
I20260320 15:27:43.554664 29640 rpcz_store.cc:276] 0320 15:27:34.114848 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.114913 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:43.379330 (+9264417us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:43.379342 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.520124 (+140782us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.526612 (+ 6488us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.554540 (+ 27928us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9254893}
I20260320 15:27:43.555140 29640 tablet_replica.cc:333] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.555321 29640 raft_consensus.cc:2243] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.555377 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet fc009401bb3d440f9d36e080efef1204 (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]) successfully deleted
I20260320 15:27:43.555485 29640 raft_consensus.cc:2272] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.555612 28734 raft_consensus.cc:359] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.555797 28734 raft_consensus.cc:385] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.555850 28734 raft_consensus.cc:740] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.556080 29640 ts_tablet_manager.cc:1916] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.556047 28734 consensus_queue.cc:260] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.554929 30313 raft_consensus.cc:515] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.557101 30313 leader_election.cc:290] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:43.557569 28382 leader_election.cc:343] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.557746 28382 leader_election.cc:304] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.558223 30344 raft_consensus.cc:2749] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.558830 28382 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.559136 30344 raft_consensus.cc:2804] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260320 15:27:43.559204 30344 raft_consensus.cc:493] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:43.559281 30344 raft_consensus.cc:3060] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:43.562227 30344 raft_consensus.cc:515] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:43.562935 30344 leader_election.cc:290] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.563210 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.565156 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.565258 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.565363 28576 raft_consensus.cc:3060] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.556547 28734 ts_tablet_manager.cc:1434] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
I20260320 15:27:43.566028 28734 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.571903 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.571981 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d92e17cea044541968356ae993be1e7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.574393 28576 raft_consensus.cc:2468] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:43.576355 28734 tablet_bootstrap.cc:654] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.577934 29640 ts_tablet_manager.cc:1929] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.578049 29640 log.cc:1199] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9176e734212849aab8d99473de02cc79
I20260320 15:27:43.578428 29640 ts_tablet_manager.cc:1950] T 9176e734212849aab8d99473de02cc79 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.578792 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 139) took 9456 ms. Trace:
I20260320 15:27:43.578881 29640 rpcz_store.cc:276] 0320 15:27:34.122111 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.122163 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:43.555079 (+9432916us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:43.555095 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.574145 (+ 19050us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.577811 (+ 3666us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.578775 (+ 964us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9430427}
I20260320 15:27:43.579027 29939 raft_consensus.cc:493] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.579138 29939 raft_consensus.cc:515] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.579306 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 137) took 9469 ms. Trace:
I20260320 15:27:43.579366 29640 rpcz_store.cc:276] 0320 15:27:34.109680 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.109743 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:43.579164 (+9469421us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:43.579173 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.579292 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9453794}
I20260320 15:27:43.579855 29939 leader_election.cc:290] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.580278 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9176e734212849aab8d99473de02cc79 (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:43.580355 28734 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.580511 28734 ts_tablet_manager.cc:1403] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
W20260320 15:27:43.580627 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 6d9b0f1936f54c699211c5ee9709f9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d9b0f1936f54c699211c5ee9709f9ed already in progress: creating tablet
I20260320 15:27:43.581321 28736 tablet_bootstrap.cc:654] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.581308 28734 raft_consensus.cc:359] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.581486 28734 raft_consensus.cc:385] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.581543 28734 raft_consensus.cc:740] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.581727 28734 consensus_queue.cc:260] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.582335 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.582859 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3bd9c79d2d74e878647f97e66a9bfca" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.583135 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.584311 28733 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.584507 28733 ts_tablet_manager.cc:1403] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.005s
I20260320 15:27:43.585486 28733 raft_consensus.cc:359] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.585688 30313 raft_consensus.cc:493] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.585772 28733 raft_consensus.cc:385] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.585861 28733 raft_consensus.cc:740] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.586052 30313 raft_consensus.cc:515] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.586114 28733 consensus_queue.cc:260] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.586690 28733 ts_tablet_manager.cc:1434] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260320 15:27:43.586984 28733 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.587026 30313 leader_election.cc:290] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.587218 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7f0242a7968541f480367b45cda5d9cc (table default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]) successfully deleted
I20260320 15:27:43.587841 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.584053 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:43.588570 28383 consensus_peers.cc:597] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 442ca186c9d4449dade39271dec5c608. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:43.590219 28383 leader_election.cc:343] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: aafd1e123cd043e7828140d2443e047f
W20260320 15:27:43.590473 28383 leader_election.cc:343] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.590605 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cb44b9cc6445cf8324cf52190d6773" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.591336 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aafd1e123cd043e7828140d2443e047f" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.591511 28576 raft_consensus.cc:3060] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Advancing to term 2
I20260320 15:27:43.593726 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cb44b9cc6445cf8324cf52190d6773" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.593904 28574 raft_consensus.cc:2468] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:43.594477 28576 raft_consensus.cc:2468] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 2.
I20260320 15:27:43.590770 28299 raft_consensus.cc:2468] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:43.596444 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.596760 28446 raft_consensus.cc:2468] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.601198 30395 consensus_queue.cc:1048] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.601089 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:43.601228 28524 leader_election.cc:304] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.603901 30395 raft_consensus.cc:2804] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.604070 30395 raft_consensus.cc:493] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.604202 30395 raft_consensus.cc:3060] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.604880 29640 tablet_replica.cc:333] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.604619 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
W20260320 15:27:43.603442 28524 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.607218 29640 raft_consensus.cc:2243] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.608826 28733 tablet_bootstrap.cc:654] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.608867 29640 raft_consensus.cc:2272] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.609333 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:43.611161 28734 ts_tablet_manager.cc:1434] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.031s user 0.002s sys 0.000s
W20260320 15:27:43.611369 28524 leader_election.cc:343] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.610993 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:43.613613 28736 tablet_bootstrap.cc:492] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:43.614151 28736 ts_tablet_manager.cc:1403] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.003s
I20260320 15:27:43.614163 29640 ts_tablet_manager.cc:1916] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.616323 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
W20260320 15:27:43.616811 28524 leader_election.cc:343] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.620062 28736 raft_consensus.cc:359] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.620298 28736 raft_consensus.cc:385] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.619719 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:43.620366 28736 raft_consensus.cc:740] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.620625 28736 consensus_queue.cc:260] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.621742 28736 ts_tablet_manager.cc:1434] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:43.621387 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:43.622077 28736 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:43.623888 28734 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.624732 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:43.625579 28734 tablet_bootstrap.cc:654] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.626953 29643 ts_tablet_manager.cc:1929] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.627082 29643 log.cc:1199] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/796c907919df47bf9369e45a1248f96b
I20260320 15:27:43.627470 29643 ts_tablet_manager.cc:1950] T 796c907919df47bf9369e45a1248f96b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.628382 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 149) took 9340 ms. Trace:
I20260320 15:27:43.628486 29643 rpcz_store.cc:276] 0320 15:27:34.288706 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.289689 (+ 983us) service_pool.cc:225] Handling call
0320 15:27:43.541999 (+9252310us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:43.542009 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.593393 (+ 51384us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.626830 (+ 33437us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.628355 (+ 1525us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9252018}
I20260320 15:27:43.628876 29643 tablet_replica.cc:333] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.629078 29643 raft_consensus.cc:2243] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.629114 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 796c907919df47bf9369e45a1248f96b (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]) successfully deleted
I20260320 15:27:43.629263 29643 raft_consensus.cc:2272] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.629844 29643 ts_tablet_manager.cc:1916] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.627889 28902 catalog_manager.cc:5671] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
W20260320 15:27:43.625655 28524 leader_election.cc:343] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.631651 28524 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.632475 28524 leader_election.cc:343] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.633240 28524 leader_election.cc:343] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.634703 28736 tablet_bootstrap.cc:654] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.634572 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:43.635704 30355 raft_consensus.cc:697] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.636022 30355 consensus_queue.cc:237] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.636119 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:43.640651 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:43.641904 30395 raft_consensus.cc:515] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.642612 30395 leader_election.cc:290] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.644551 28733 tablet_bootstrap.cc:492] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.644716 28733 ts_tablet_manager.cc:1403] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.001s
I20260320 15:27:43.645630 28733 raft_consensus.cc:359] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.645828 28733 raft_consensus.cc:385] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.645888 28733 raft_consensus.cc:740] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.646178 28733 consensus_queue.cc:260] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.646716 28733 ts_tablet_manager.cc:1434] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:43.647024 28733 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.648214 28902 catalog_manager.cc:5671] T 01078353a3a343a3a1231245a391cf8e P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:43.653069 28733 tablet_bootstrap.cc:654] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.654386 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:43.659703 29678 ts_tablet_manager.cc:1929] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.659931 29678 log.cc:1199] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/583ed053b2a742c7b029f348f4495101
I20260320 15:27:43.660353 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
W20260320 15:27:43.661082 28524 leader_election.cc:343] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.662503 28524 leader_election.cc:343] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.664376 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:43.664733 29640 ts_tablet_manager.cc:1929] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:43.665572 29640 log.cc:1199] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/56fa691a26114657ad768545a30dfd3c
I20260320 15:27:43.666193 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.666390 28443 raft_consensus.cc:3060] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.666391 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6197f6a1e1cf4007a481be3d7ccfd425" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.667691 29640 ts_tablet_manager.cc:1950] T 56fa691a26114657ad768545a30dfd3c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:43.669239 28443 raft_consensus.cc:2468] T 6197f6a1e1cf4007a481be3d7ccfd425 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.669684 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 140) took 9542 ms. Trace:
I20260320 15:27:43.669884 30407 raft_consensus.cc:493] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:43.670023 30407 raft_consensus.cc:515] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:43.670712 30407 leader_election.cc:290] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:43.670100 29640 rpcz_store.cc:276] 0320 15:27:34.127358 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.127414 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:43.604535 (+9477121us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:43.604549 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.637256 (+ 32707us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.664136 (+ 26880us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.669507 (+ 5371us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9473154}
I20260320 15:27:43.671705 28733 tablet_bootstrap.cc:492] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:43.671669 28239 leader_election.cc:343] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:43.671738 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 56fa691a26114657ad768545a30dfd3c (table default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]) successfully deleted
I20260320 15:27:43.671902 28733 ts_tablet_manager.cc:1403] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20260320 15:27:43.672501 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 141) took 9530 ms. Trace:
I20260320 15:27:43.672763 28733 raft_consensus.cc:359] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:43.673404 28240 leader_election.cc:343] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:43.673527 28240 leader_election.cc:304] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.673831 28734 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.672916 29640 rpcz_store.cc:276] 0320 15:27:34.144424 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.144485 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:43.671726 (+9527241us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:43.671899 (+ 173us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.672325 (+ 426us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9526954}
I20260320 15:27:43.673969 28734 ts_tablet_manager.cc:1403] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.003s
I20260320 15:27:43.674165 30407 raft_consensus.cc:2749] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:43.674698 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dd315b5d49104d679b03515b65ee30ac already in progress: creating tablet
I20260320 15:27:43.674894 28734 raft_consensus.cc:359] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.675099 28734 raft_consensus.cc:385] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.675158 28734 raft_consensus.cc:740] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.675352 28734 consensus_queue.cc:260] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.675587 29640 tablet_replica.cc:333] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:43.675872 28734 ts_tablet_manager.cc:1434] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:43.675184 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:43.676169 28734 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.677753 28734 tablet_bootstrap.cc:654] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.660742 29678 ts_tablet_manager.cc:1950] T 583ed053b2a742c7b029f348f4495101 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.678676 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:43.681707 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:43.683486 28736 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:43.684005 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:43.684832 29643 ts_tablet_manager.cc:1929] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:43.684963 29643 log.cc:1199] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:43.685060 28736 ts_tablet_manager.cc:1403] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.063s user 0.000s sys 0.007s
I20260320 15:27:43.685377 29643 ts_tablet_manager.cc:1950] T 5438fdc6e6944e79ac8aecd0ef774467 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.685750 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 150) took 9394 ms. Trace:
I20260320 15:27:43.685842 29643 rpcz_store.cc:276] 0320 15:27:34.292443 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.292696 (+ 253us) service_pool.cc:225] Handling call
0320 15:27:43.628820 (+9336124us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:43.628832 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.661363 (+ 32531us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.684694 (+ 23331us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.685728 (+ 1034us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9335831}
W20260320 15:27:43.684620 28524 leader_election.cc:343] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.686246 29643 tablet_replica.cc:333] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.685920 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:43.686498 29643 raft_consensus.cc:2243] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.686444 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 5438fdc6e6944e79ac8aecd0ef774467 (table default.Table_38cfb7d2f6bf45179c9e84b90d74a2f1 [id=b4139eb7399c448db53ff05dd07f7a40]) successfully deleted
I20260320 15:27:43.686857 29643 raft_consensus.cc:2272] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.686398 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 158) took 9388 ms. Trace:
I20260320 15:27:43.687466 29643 ts_tablet_manager.cc:1916] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.687472 28736 raft_consensus.cc:359] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.688771 28736 raft_consensus.cc:385] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.688999 28736 raft_consensus.cc:740] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.689592 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:43.687717 29678 rpcz_store.cc:276] 0320 15:27:34.297393 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.299550 (+ 2157us) service_pool.cc:225] Handling call
0320 15:27:43.552163 (+9252613us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:43.552173 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.620127 (+ 67954us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.659376 (+ 39249us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.685987 (+ 26611us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9252154}
I20260320 15:27:43.690865 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:43.691645 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 583ed053b2a742c7b029f348f4495101 (table default.Table_7c20a59bd0eb41739dfa0770f3cff465 [id=230e2221091144ffbc8f90bbf2c03ded]) successfully deleted
I20260320 15:27:43.672948 28733 raft_consensus.cc:385] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.692636 28733 raft_consensus.cc:740] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.691421 28736 consensus_queue.cc:260] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.694132 29678 tablet_replica.cc:333] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.694617 29678 raft_consensus.cc:2243] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.693934 28733 consensus_queue.cc:260] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.695220 28736 ts_tablet_manager.cc:1434] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:43.696184 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:43.697803 28733 ts_tablet_manager.cc:1434] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.026s user 0.003s sys 0.000s
I20260320 15:27:43.698437 28524 leader_election.cc:304] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.699590 28733 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.700268 28736 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:43.701507 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:43.700506 28734 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.703205 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:43.703727 28734 ts_tablet_manager.cc:1403] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.001s
I20260320 15:27:43.704604 28734 raft_consensus.cc:359] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.704800 28734 raft_consensus.cc:385] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.704856 28734 raft_consensus.cc:740] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.705055 28734 consensus_queue.cc:260] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.705601 28734 ts_tablet_manager.cc:1434] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:43.705907 28734 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.706681 30355 raft_consensus.cc:493] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.706933 30355 raft_consensus.cc:515] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.707450 28734 tablet_bootstrap.cc:654] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.707823 30355 leader_election.cc:290] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:43.708262 30355 raft_consensus.cc:2804] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.708546 30355 raft_consensus.cc:697] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.708841 30355 consensus_queue.cc:237] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.708706 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:43.710959 28733 tablet_bootstrap.cc:654] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.711378 28736 tablet_bootstrap.cc:654] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.711511 28902 catalog_manager.cc:5671] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:43.714833 30434 consensus_queue.cc:1048] T ec6961acec774d2f9161f057f64b3fdf P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.715322 29678 raft_consensus.cc:2272] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.716018 29678 ts_tablet_manager.cc:1916] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.716353 28904 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_07e18d8b90c34136804ef8e64f659f82 [id=90afb14544844334aff0e82fa156d6d9] alter complete (version 2)
W20260320 15:27:43.716487 28903 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf37dbb8bc8e48c5a11f106ecd93dd78 consensus state has no leader
W20260320 15:27:43.716605 28906 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 321fe367ec3348f8a049167f3f8c15d5 consensus state has no leader
I20260320 15:27:43.716730 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 97) took 4540 ms. Trace:
I20260320 15:27:43.716886 28906 rpcz_store.cc:276] 0320 15:27:39.176326 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.597472 (+2421146us) service_pool.cc:225] Handling call
0320 15:27:41.597963 (+ 491us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:41.598255 (+ 292us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:41.684720 (+ 86465us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:41.684878 (+ 158us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:41.795681 (+110803us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:43.716685 (+1921004us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:41.685648 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:41.685785 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:41.685794 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:41.685798 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:41.685916 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:41.685931 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:41.685936 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:41.686004 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:41.686006 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:41.686044 (+ 38us) write_op.cc:270] Start()
0320 15:27:41.686082 (+ 38us) write_op.cc:276] Timestamp: P: 1774020461686032 usec, L: 0
0320 15:27:41.686086 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:41.686311 (+ 225us) log.cc:844] Serialized 405 byte log entry
0320 15:27:41.786034 (+ 99723us) op_driver.cc:464] REPLICATION: finished
0320 15:27:41.792880 (+ 6846us) write_op.cc:301] APPLY: starting
0320 15:27:41.792903 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:41.792959 (+ 56us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:41.793021 (+ 62us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:41.793103 (+ 82us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:41.793142 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:41.793151 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:41.793335 (+ 184us) log.cc:844] Serialized 23 byte log entry
0320 15:27:41.795523 (+ 2188us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.795573 (+ 50us) write_op.cc:454] Released schema lock
0320 15:27:41.795583 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:41.795676 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.795679 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6738,"apply.run_cpu_time_us":721,"apply.run_wall_time_us":2799,"num_ops":1,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":751,"prepare.run_wall_time_us":752,"replication_time_us":99912}]]}
W20260320 15:27:43.717839 28906 rpcz_store.cc:267] Call kudu.master.MasterService.Ping from 127.27.74.1:41749 (request call id 122) took 804 ms (client timeout 1000 ms). Trace:
W20260320 15:27:43.717986 28906 rpcz_store.cc:269] 0320 15:27:42.913384 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.717712 (+804328us) service_pool.cc:225] Handling call
0320 15:27:43.717785 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:43.718650 28906 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f4751ac3d9fd417ba487d086d653b85d" } modify_external_catalogs: true
I20260320 15:27:43.718897 28906 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f4751ac3d9fd417ba487d086d653b85d" } modify_external_catalogs: true
I20260320 15:27:43.719727 28350 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:43.719928 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:43.716735 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 98) took 4534 ms. Trace:
I20260320 15:27:43.721841 28903 rpcz_store.cc:276] 0320 15:27:39.182584 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.679336 (+2496752us) service_pool.cc:225] Handling call
0320 15:27:41.685715 (+ 6379us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:41.685935 (+ 220us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:41.686073 (+ 138us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:41.686150 (+ 77us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:41.810249 (+124099us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:43.716658 (+1906409us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:41.686664 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:41.686753 (+ 89us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:41.686759 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:41.686761 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:41.686890 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:41.686906 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:41.686914 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:41.686987 (+ 73us) tablet.cc:700] Row locks acquired
0320 15:27:41.686992 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:41.687028 (+ 36us) write_op.cc:270] Start()
0320 15:27:41.687068 (+ 40us) write_op.cc:276] Timestamp: P: 1774020461687016 usec, L: 0
0320 15:27:41.687073 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:41.687256 (+ 183us) log.cc:844] Serialized 405 byte log entry
0320 15:27:41.786124 (+ 98868us) op_driver.cc:464] REPLICATION: finished
0320 15:27:41.794507 (+ 8383us) write_op.cc:301] APPLY: starting
0320 15:27:41.794526 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:41.794575 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:41.794642 (+ 67us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:41.794734 (+ 92us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:41.794766 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:41.794774 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:41.794939 (+ 165us) log.cc:844] Serialized 23 byte log entry
0320 15:27:41.808771 (+ 13832us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.808801 (+ 30us) write_op.cc:454] Released schema lock
0320 15:27:41.808812 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:41.808931 (+ 119us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.808936 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8312,"apply.run_cpu_time_us":738,"apply.run_wall_time_us":14406,"num_ops":1,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":655,"prepare.run_wall_time_us":654,"replication_time_us":99026,"thread_start_us":7465,"threads_started":1}]]}
I20260320 15:27:43.722746 28903 master_service.cc:954] ReplaceTablet: received request to replace tablet 06cb44b9cc6445cf8324cf52190d6773 from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:43.735793 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 145310us
I20260320 15:27:43.737326 30155 consensus_queue.cc:1048] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.739887 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740098 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740276 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740432 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740592 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e7b964395f24e45bc2715be3fac6dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740765 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740918 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.740983 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.744367 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.746132 28493 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:43.746301 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:43.746788 29643 ts_tablet_manager.cc:1929] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.746898 29643 log.cc:1199] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/442ca186c9d4449dade39271dec5c608
I20260320 15:27:43.747282 29643 ts_tablet_manager.cc:1950] T 442ca186c9d4449dade39271dec5c608 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.747645 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 151) took 9450 ms. Trace:
I20260320 15:27:43.751562 28382 leader_election.cc:304] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.751749 28734 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:43.751868 28734 ts_tablet_manager.cc:1403] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.000s
W20260320 15:27:43.751904 28382 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 157342us
I20260320 15:27:43.752131 28382 leader_election.cc:304] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.752403 28382 leader_election.cc:304] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.753024 28734 raft_consensus.cc:359] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.753218 28734 raft_consensus.cc:385] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.753506 28734 raft_consensus.cc:740] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.753716 28734 consensus_queue.cc:260] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.753201 30344 raft_consensus.cc:2804] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.755368 29678 ts_tablet_manager.cc:1929] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.755502 29678 log.cc:1199] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:43.755793 30427 raft_consensus.cc:2804] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.755877 30427 raft_consensus.cc:493] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.755939 30427 raft_consensus.cc:3060] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.756552 29678 ts_tablet_manager.cc:1950] T 5fa5f6fe801b4452bf294a9ae23bb163 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.757444 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 159) took 9457 ms. Trace:
I20260320 15:27:43.757539 29678 rpcz_store.cc:276] 0320 15:27:34.299866 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.300053 (+ 187us) service_pool.cc:225] Handling call
0320 15:27:43.693139 (+9393086us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:43.693525 (+ 386us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.729376 (+ 35851us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.755222 (+ 25846us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.757420 (+ 2198us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9392143}
I20260320 15:27:43.758956 30427 raft_consensus.cc:515] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.747722 29643 rpcz_store.cc:276] 0320 15:27:34.298679 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.298860 (+ 181us) service_pool.cc:225] Handling call
0320 15:27:43.686175 (+9387315us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:43.686185 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.714366 (+ 28181us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.746672 (+ 32306us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.747628 (+ 956us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9386935}
I20260320 15:27:43.764042 29643 tablet_replica.cc:333] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.764207 30256 raft_consensus.cc:493] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.764451 30256 raft_consensus.cc:515] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.765153 30256 leader_election.cc:290] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:43.768465 28240 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:43.768913 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 442ca186c9d4449dade39271dec5c608 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
I20260320 15:27:43.769230 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5fa5f6fe801b4452bf294a9ae23bb163 (table default.Table_1fdbd3e87ebe4dcc82bdd58cb155085c [id=949da1fc47384bf39686c98425dfdd45]) successfully deleted
I20260320 15:27:43.769733 30313 raft_consensus.cc:2804] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260320 15:27:43.771962 29678 tablet_replica.cc:333] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.772220 29678 raft_consensus.cc:2243] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.772416 29678 raft_consensus.cc:2272] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.773036 29678 ts_tablet_manager.cc:1916] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.773103 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd315b5d49104d679b03515b65ee30ac" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.773339 28576 raft_consensus.cc:2468] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:43.773895 28239 leader_election.cc:304] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.774281 30407 raft_consensus.cc:2804] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.774371 30407 raft_consensus.cc:493] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.774448 30407 raft_consensus.cc:3060] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.777245 30407 raft_consensus.cc:515] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.778451 28733 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:43.780028 28733 ts_tablet_manager.cc:1403] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.081s user 0.009s sys 0.000s
W20260320 15:27:43.780783 28240 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:43.781690 30155 raft_consensus.cc:493] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.781845 30155 raft_consensus.cc:515] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.782438 30155 leader_election.cc:290] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.783233 28634 heartbeater.cc:344] Connected to a master server at 127.27.74.62:43951
I20260320 15:27:43.783403 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:43.788215 30457 raft_consensus.cc:493] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.788406 30457 raft_consensus.cc:515] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.789129 30457 leader_election.cc:290] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.790434 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:43.791049 28383 leader_election.cc:343] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.789889 28733 raft_consensus.cc:359] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.792326 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.792575 28444 raft_consensus.cc:2468] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:43.792657 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.791800 28733 raft_consensus.cc:385] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:43.793148 28733 raft_consensus.cc:740] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:43.794497 28524 leader_election.cc:304] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.794961 30155 raft_consensus.cc:2804] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.795028 30155 raft_consensus.cc:493] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.795107 30155 raft_consensus.cc:3060] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.796294 28903 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:43.796916 28903 catalog_manager.cc:6741] ReplaceTablet: tablet 06cb44b9cc6445cf8324cf52190d6773 deleted and replaced by tablet 91f393d2ca304b899f788add43da7e9a
I20260320 15:27:43.797052 28903 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 113) took 2975 ms. Trace:
I20260320 15:27:43.797173 28903 rpcz_store.cc:276] 0320 15:27:40.821951 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.722620 (+2900669us) service_pool.cc:225] Handling call
0320 15:27:43.797012 (+ 74392us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.746397 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.746591 (+ 194us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.746598 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:43.746601 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:43.746726 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.746742 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.746749 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:43.746835 (+ 86us) tablet.cc:700] Row locks acquired
0320 15:27:43.746838 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:43.746875 (+ 37us) write_op.cc:270] Start()
0320 15:27:43.746912 (+ 37us) write_op.cc:276] Timestamp: P: 1774020463746861 usec, L: 0
0320 15:27:43.746916 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.747115 (+ 199us) log.cc:844] Serialized 622 byte log entry
0320 15:27:43.794947 (+ 47832us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.795325 (+ 378us) write_op.cc:301] APPLY: starting
0320 15:27:43.795351 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.795407 (+ 56us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.795442 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.795632 (+ 190us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.795707 (+ 75us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.795717 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:43.795836 (+ 119us) log.cc:844] Serialized 29 byte log entry
0320 15:27:43.795930 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.795958 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:43.795968 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.796067 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.796071 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":300,"apply.run_cpu_time_us":725,"apply.run_wall_time_us":735,"num_ops":2,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":792,"prepare.run_wall_time_us":791,"replication_time_us":47982}]]}
I20260320 15:27:43.798295 30155 raft_consensus.cc:515] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.798885 30155 leader_election.cc:290] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.800141 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 1bd3032950254e6f9db8da35dbb1acc5
I20260320 15:27:43.801052 28906 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 112) took 3252 ms. Trace:
I20260320 15:27:43.801182 28906 rpcz_store.cc:276] 0320 15:27:40.548883 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.718443 (+3169560us) service_pool.cc:225] Handling call
0320 15:27:43.719015 (+ 572us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:43.719234 (+ 219us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:43.719321 (+ 87us) catalog_manager.cc:2824] Locking tablets
0320 15:27:43.747926 (+ 28605us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:43.799901 (+ 51975us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:43.800017 (+ 116us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:43.800071 (+ 54us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:43.800969 (+ 898us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.748743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.748869 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.748875 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:43.748877 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:43.748990 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.749002 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.749007 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:43.749108 (+ 101us) tablet.cc:700] Row locks acquired
0320 15:27:43.749110 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:43.749144 (+ 34us) write_op.cc:270] Start()
0320 15:27:43.749178 (+ 34us) write_op.cc:276] Timestamp: P: 1774020463749130 usec, L: 0
0320 15:27:43.749182 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.749341 (+ 159us) log.cc:844] Serialized 759 byte log entry
0320 15:27:43.795061 (+ 45720us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.796172 (+ 1111us) write_op.cc:301] APPLY: starting
0320 15:27:43.796185 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.796227 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.796283 (+ 56us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.796358 (+ 75us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.796388 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:43.796394 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:43.796463 (+ 69us) log.cc:844] Serialized 29 byte log entry
0320 15:27:43.796523 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.796552 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:43.796558 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1073,"apply.run_cpu_time_us":449,"apply.run_wall_time_us":448,"num_ops":2,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":674,"prepare.run_wall_time_us":674,"replication_time_us":45834,"threads_started":1}]]}
I20260320 15:27:43.802206 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:43.802657 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 with delete_type TABLET_DATA_DELETED (replaced by tablet 91f393d2ca304b899f788add43da7e9a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:43.808548 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 with delete_type TABLET_DATA_DELETED (replaced by tablet 91f393d2ca304b899f788add43da7e9a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.808663 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:43 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.808828 29678 ts_tablet_manager.cc:1929] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.808986 29678 log.cc:1199] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:43.809137 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 with delete_type TABLET_DATA_DELETED (replaced by tablet 91f393d2ca304b899f788add43da7e9a) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:43.808969 28524 consensus_peers.cc:597] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ef09e4afdd1f4fce9caec4dcafa2c50c. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.809372 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:43 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.809415 29678 ts_tablet_manager.cc:1950] T ea6ff59d2ffe4d579e6db2375eb11414 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.809643 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:43 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:43.810936 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 160) took 9510 ms. Trace:
I20260320 15:27:43.811030 29678 rpcz_store.cc:276] 0320 15:27:34.300002 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.300279 (+ 277us) service_pool.cc:225] Handling call
0320 15:27:43.757824 (+9457545us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:43.757837 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.776437 (+ 18600us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.779424 (+ 2987us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.810910 (+ 31486us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9457342}
I20260320 15:27:43.811471 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ea6ff59d2ffe4d579e6db2375eb11414 (table default.Table_adb981152b5c4e7793de15d294bb7f34 [id=db8746d23d9e45f681e9cc2b413b8b3b]) successfully deleted
I20260320 15:27:43.812260 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.812454 28444 raft_consensus.cc:3060] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.814021 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.814697 29678 tablet_replica.cc:333] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.815085 29678 raft_consensus.cc:2243] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 2 LEADER]: Raft consensus shutting down.
I20260320 15:27:43.815567 29678 raft_consensus.cc:2272] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.816187 29678 ts_tablet_manager.cc:1916] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.816887 28444 raft_consensus.cc:2468] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:43.795601 28733 consensus_queue.cc:260] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.819132 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.824240 28733 ts_tablet_manager.cc:1434] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.044s user 0.002s sys 0.002s
W20260320 15:27:43.820691 28524 leader_election.cc:343] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.833856 30256 raft_consensus.cc:493] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.834039 30256 raft_consensus.cc:515] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.834769 30256 leader_election.cc:290] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:43.835265 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:43.835477 28574 raft_consensus.cc:2468] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:43.835821 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.836009 28445 raft_consensus.cc:2468] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:43.836440 28240 leader_election.cc:304] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.836958 29678 ts_tablet_manager.cc:1929] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:43.837085 29678 log.cc:1199] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:43.837240 30256 raft_consensus.cc:2804] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.837333 30256 raft_consensus.cc:493] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.837404 30256 raft_consensus.cc:3060] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.837472 29678 ts_tablet_manager.cc:1950] T 8dc27a4968ca4203b67dd3064fb03868 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.838173 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7fb1733e7a0e4774a5be2e06208bb495" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.838241 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 157) took 9541 ms. Trace:
I20260320 15:27:43.838421 28446 raft_consensus.cc:2393] T 7fb1733e7a0e4774a5be2e06208bb495 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:43.838322 29678 rpcz_store.cc:276] 0320 15:27:34.297272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.297545 (+ 273us) service_pool.cc:225] Handling call
0320 15:27:43.814630 (+9517085us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:43.814644 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.833622 (+ 18978us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.836840 (+ 3218us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.838222 (+ 1382us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9508624}
I20260320 15:27:43.838841 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8dc27a4968ca4203b67dd3064fb03868 (table default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]) successfully deleted
W20260320 15:27:43.838956 28382 leader_election.cc:343] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.839059 28382 leader_election.cc:304] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.840155 30256 raft_consensus.cc:515] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.839918 30461 raft_consensus.cc:2749] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.841117 30427 leader_election.cc:290] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.841379 30256 leader_election.cc:290] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:43.841643 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:43.841817 28444 raft_consensus.cc:3060] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.841887 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ccfd2d5e82448b99f1022954b98c6cc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.842075 28574 raft_consensus.cc:3060] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.843755 29643 raft_consensus.cc:2243] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.843961 29643 raft_consensus.cc:2272] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.844533 29643 ts_tablet_manager.cc:1916] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.845147 28444 raft_consensus.cc:2468] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:43.845626 28240 leader_election.cc:304] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.846000 30256 raft_consensus.cc:2804] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.846143 30256 raft_consensus.cc:697] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:43.846405 30256 consensus_queue.cc:237] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.848210 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cb44b9cc6445cf8324cf52190d6773" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:43.848490 28297 raft_consensus.cc:3060] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.851038 28297 raft_consensus.cc:2468] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:43.852258 28383 leader_election.cc:304] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:43.849010 28574 raft_consensus.cc:2468] T 1ccfd2d5e82448b99f1022954b98c6cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:43.854394 29643 ts_tablet_manager.cc:1929] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.854609 29643 log.cc:1199] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/906f207dee674a4288c8c60b5d3ff9f8
I20260320 15:27:43.855305 29643 ts_tablet_manager.cc:1950] T 906f207dee674a4288c8c60b5d3ff9f8 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.856292 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06cb44b9cc6445cf8324cf52190d6773" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.856526 28574 raft_consensus.cc:3060] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.855487 30427 raft_consensus.cc:2804] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.855798 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 152) took 9555 ms. Trace:
I20260320 15:27:43.860443 29643 rpcz_store.cc:276] 0320 15:27:34.303021 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.303587 (+ 566us) service_pool.cc:225] Handling call
0320 15:27:43.763978 (+9460391us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:43.763995 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.849268 (+ 85273us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.854252 (+ 4984us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.855773 (+ 1521us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":79444,"tablet-delete.queue_time_us":9460135}
I20260320 15:27:43.861050 29643 tablet_replica.cc:333] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:43.860807 28524 leader_election.cc:343] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.861408 29643 raft_consensus.cc:2243] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.861529 30407 leader_election.cc:290] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:43.861713 29643 raft_consensus.cc:2272] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.860260 30427 raft_consensus.cc:697] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:43.862358 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd315b5d49104d679b03515b65ee30ac" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:43.862540 28576 raft_consensus.cc:3060] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.862646 30427 consensus_queue.cc:237] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.863231 30359 raft_consensus.cc:493] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.862797 29643 ts_tablet_manager.cc:1916] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.863366 30359 raft_consensus.cc:515] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.863965 30359 leader_election.cc:290] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:43.865797 29678 tablet_replica.cc:333] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.866405 28576 raft_consensus.cc:2468] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:43.866742 29678 raft_consensus.cc:2243] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.866835 28239 leader_election.cc:304] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.867046 29678 raft_consensus.cc:2272] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.867103 30407 raft_consensus.cc:2804] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.867830 30407 raft_consensus.cc:697] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:43.868132 30407 consensus_queue.cc:237] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:43.868645 29678 ts_tablet_manager.cc:1916] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.871630 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.873980 28524 leader_election.cc:304] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:43.874323 30155 raft_consensus.cc:2804] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:43.874452 30155 raft_consensus.cc:697] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:43.874789 30155 consensus_queue.cc:237] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.884675 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 7394e941c99e4358ad7a33db84b15348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.884967 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.885066 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.885324 28574 raft_consensus.cc:2468] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:43.887694 28524 leader_election.cc:304] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:43.888103 30395 raft_consensus.cc:3060] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.891049 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.891072 30395 raft_consensus.cc:2749] T 7fb1733e7a0e4774a5be2e06208bb495 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:43.899497 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.899942 29643 ts_tablet_manager.cc:1929] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:43.903785 29643 log.cc:1199] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7f0242a7968541f480367b45cda5d9cc
I20260320 15:27:43.903939 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 906f207dee674a4288c8c60b5d3ff9f8 (table default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]) successfully deleted
I20260320 15:27:43.904366 29643 ts_tablet_manager.cc:1950] T 7f0242a7968541f480367b45cda5d9cc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.904491 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:43.904768 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 153) took 9600 ms. Trace:
I20260320 15:27:43.904866 29643 rpcz_store.cc:276] 0320 15:27:34.305542 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.305793 (+ 251us) service_pool.cc:225] Handling call
0320 15:27:43.860988 (+9555195us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:43.861003 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.876203 (+ 15200us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.899816 (+ 23613us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.904751 (+ 4935us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9554977}
I20260320 15:27:43.905210 29643 tablet_replica.cc:333] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.905223 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.905427 29643 raft_consensus.cc:2243] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.905625 29643 raft_consensus.cc:2272] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.906252 29643 ts_tablet_manager.cc:1916] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.906787 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.909614 29678 ts_tablet_manager.cc:1929] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:43.909739 29678 log.cc:1199] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ad4a30476e7146afae1e77dad078136e
I20260320 15:27:43.911314 29678 ts_tablet_manager.cc:1950] T ad4a30476e7146afae1e77dad078136e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:43.912225 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 156) took 9616 ms. Trace:
I20260320 15:27:43.912324 29678 rpcz_store.cc:276] 0320 15:27:34.297107 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.297575 (+ 468us) service_pool.cc:225] Handling call
0320 15:27:43.865733 (+9568158us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:43.865746 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.892535 (+ 26789us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.909480 (+ 16945us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.912205 (+ 2725us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9558010}
I20260320 15:27:43.915025 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.923962 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:43.927075 28524 leader_election.cc:343] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:43.927783 28524 consensus_peers.cc:597] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.933526 30394 consensus_queue.cc:1048] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:43.934509 29678 tablet_replica.cc:333] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:43.934580 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ad4a30476e7146afae1e77dad078136e (table default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]) successfully deleted
I20260320 15:27:43.934744 29678 raft_consensus.cc:2243] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.934942 29678 raft_consensus.cc:2272] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.935493 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.935918 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.936553 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.937649 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.938442 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:43.939108 28892 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 105) took 3906 ms (client timeout 1000 ms). Trace:
I20260320 15:27:43.940106 29678 ts_tablet_manager.cc:1916] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.943742 28733 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:43.945652 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.946061 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.946372 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.946990 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.947329 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.947729 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet b6201b74fdb0428e98f2eebb0b24d01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.951802 29640 raft_consensus.cc:2243] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.951898 28734 ts_tablet_manager.cc:1434] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.200s user 0.014s sys 0.028s
I20260320 15:27:43.952061 29640 raft_consensus.cc:2272] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.952508 28734 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:43.952786 29640 ts_tablet_manager.cc:1916] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:43.954684 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.955119 30313 raft_consensus.cc:697] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:43.955447 30313 consensus_queue.cc:237] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:43.956118 28734 tablet_bootstrap.cc:654] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:43.957314 30344 raft_consensus.cc:697] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:43.957679 30344 consensus_queue.cc:237] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.965247 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:43.966361 28239 consensus_peers.cc:597] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8241a342054240d2ac62b729cc9c9a62. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:43.969297 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.969604 30257 raft_consensus.cc:493] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:43.969740 30257 raft_consensus.cc:515] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.970379 30257 leader_election.cc:290] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:43.971498 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:43.972209 28239 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:43.972774 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:43.973763 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.974001 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.974220 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.974493 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.974727 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 757b5afdea674217909610e3719a8804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975016 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 583ed053b2a742c7b029f348f4495101 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975476 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975660 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975699 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a18f32c29224bd883ceb20658dff228 with delete_type TABLET_DATA_DELETED (replaced by tablet 7fb1733e7a0e4774a5be2e06208bb495) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975852 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.975996 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976157 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976290 28274 tablet_service.cc:1558] Processing DeleteTablet for tablet 5438fdc6e6944e79ac8aecd0ef774467 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976476 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 796c907919df47bf9369e45a1248f96b with delete_type TABLET_DATA_DELETED (replaced by tablet 6bbc5d5f49a04216a352ddb1b571dce0) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976639 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976869 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.976969 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.977102 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.977303 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.979993 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:43.978894 28443 raft_consensus.cc:2468] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:43.981709 28240 leader_election.cc:304] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:43.982580 30256 raft_consensus.cc:2804] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:43.983045 30256 raft_consensus.cc:493] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:43.983582 30256 raft_consensus.cc:3060] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:43.978940 29643 ts_tablet_manager.cc:1929] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:43.987493 29643 log.cc:1199] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:43.987910 29643 ts_tablet_manager.cc:1950] T adb65832eefe4bd4b08d95aa83c7bd7b P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:43.989008 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 154) took 9682 ms. Trace:
I20260320 15:27:43.989130 29643 rpcz_store.cc:276] 0320 15:27:34.307662 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.307955 (+ 293us) service_pool.cc:225] Handling call
0320 15:27:43.905164 (+9597209us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:43.905172 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.930083 (+ 24911us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.978828 (+ 48745us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:43.988979 (+ 10151us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9596974}
I20260320 15:27:43.989506 29643 tablet_replica.cc:333] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:43.989851 29643 raft_consensus.cc:2243] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:43.990108 29643 raft_consensus.cc:2272] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:43.992822 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.993227 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:43.993615 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.000602 30257 consensus_queue.cc:1048] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.011658 28736 tablet_bootstrap.cc:492] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.011852 28736 ts_tablet_manager.cc:1403] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.312s user 0.006s sys 0.001s
I20260320 15:27:44.012759 28736 raft_consensus.cc:359] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.012972 28736 raft_consensus.cc:385] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.013039 28736 raft_consensus.cc:740] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.013371 28736 consensus_queue.cc:260] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:43.969666 28902 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_cbe58272fb864ea085279b8bbf91db9a [id=672482e082054df5a37f0eb4413aa8fc] alter complete (version 1)
W20260320 15:27:44.014200 28892 rpcz_store.cc:269] 0320 15:27:40.032320 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.277940 (+245620us) service_pool.cc:225] Handling call
0320 15:27:43.939005 (+3661065us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":83515,"reported_tablets":190}
I20260320 15:27:43.998337 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec
I20260320 15:27:44.017567 28906 catalog_manager.cc:6741] ReplaceTablet: tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec deleted and replaced by tablet 6049ed1c2d84494893bfa20a9d563838
I20260320 15:27:44.017720 28906 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 114) took 3119 ms. Trace:
I20260320 15:27:44.017859 28906 rpcz_store.cc:276] 0320 15:27:40.898291 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.802080 (+2903789us) service_pool.cc:225] Handling call
0320 15:27:43.961740 (+159660us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:44.017679 (+ 55939us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.963373 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.963513 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.963520 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:43.963523 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:43.963684 (+ 161us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.963697 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.963702 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:43.963790 (+ 88us) tablet.cc:700] Row locks acquired
0320 15:27:43.963792 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:43.963828 (+ 36us) write_op.cc:270] Start()
0320 15:27:43.963864 (+ 36us) write_op.cc:276] Timestamp: P: 1774020463963796 usec, L: 0
0320 15:27:43.963868 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.964042 (+ 174us) log.cc:844] Serialized 622 byte log entry
0320 15:27:43.968947 (+ 4905us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.969560 (+ 613us) write_op.cc:301] APPLY: starting
0320 15:27:43.969572 (+ 12us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.969604 (+ 32us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.969630 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.969731 (+ 101us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.969783 (+ 52us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.969789 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:43.969845 (+ 56us) log.cc:844] Serialized 29 byte log entry
0320 15:27:43.969884 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.969900 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:43.969907 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.969974 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.969978 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":366065024,"child_traces":[["op",{"apply.queue_time_us":581,"apply.run_cpu_time_us":404,"apply.run_wall_time_us":404,"num_ops":2,"prepare.queue_time_us":557,"prepare.run_cpu_time_us":766,"prepare.run_wall_time_us":765,"raft.queue_time_us":578,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":5061}]]}
I20260320 15:27:44.018683 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 27ccc693e7704650bd77e0c4ca11862e from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:44.019029 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec with delete_type TABLET_DATA_DELETED (replaced by tablet 6049ed1c2d84494893bfa20a9d563838) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.019966 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec with delete_type TABLET_DATA_DELETED (replaced by tablet 6049ed1c2d84494893bfa20a9d563838) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.020602 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.021044 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.021791 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.028887 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:44.031404 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 119) took 4395 ms (client timeout 1000 ms). Trace:
W20260320 15:27:44.031649 28905 rpcz_store.cc:269] 0320 15:27:39.635721 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.649963 (+ 14242us) service_pool.cc:225] Handling call
0320 15:27:44.031346 (+4381383us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:40.340688 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:40.340885 (+ 197us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:40.340897 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:40.340902 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:40.341139 (+ 237us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:40.341161 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:40.341170 (+ 9us) tablet.cc:684] Acquiring locks for 4 operations
0320 15:27:40.348301 (+ 7131us) tablet.cc:700] Row locks acquired
0320 15:27:40.348307 (+ 6us) write_op.cc:260] PREPARE: finished
0320 15:27:40.348376 (+ 69us) write_op.cc:270] Start()
0320 15:27:40.348430 (+ 54us) write_op.cc:276] Timestamp: P: 1774020460348335 usec, L: 0
0320 15:27:40.348437 (+ 7us) op_driver.cc:348] REPLICATION: starting
0320 15:27:40.348658 (+ 221us) log.cc:844] Serialized 1650 byte log entry
0320 15:27:40.355189 (+ 6531us) op_driver.cc:464] REPLICATION: finished
0320 15:27:40.355343 (+ 154us) write_op.cc:301] APPLY: starting
0320 15:27:40.355369 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:40.355493 (+ 124us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:40.355566 (+ 73us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:40.355709 (+ 143us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:40.355762 (+ 53us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0320 15:27:40.355769 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:40.356666 (+ 897us) log.cc:844] Serialized 41 byte log entry
0320 15:27:40.356748 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.356776 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:40.356792 (+ 16us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:40.356885 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:40.356888 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":83268,"reported_tablets":197,"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":855,"apply.run_wall_time_us":1531,"num_ops":4,"prepare.queue_time_us":70,"prepare.run_cpu_time_us":1226,"prepare.run_wall_time_us":8107,"raft.queue_time_us":480,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":2653,"replication_time_us":6703}]]}
I20260320 15:27:44.034127 28733 tablet_bootstrap.cc:654] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.040581 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.040800 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.040845 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.040987 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.042582 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.044682 28736 ts_tablet_manager.cc:1434] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.033s user 0.002s sys 0.000s
I20260320 15:27:44.045080 28736 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.049688 30257 consensus_queue.cc:1048] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.049973 29678 ts_tablet_manager.cc:1929] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.050086 29678 log.cc:1199] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/906f207dee674a4288c8c60b5d3ff9f8
I20260320 15:27:44.051175 29678 ts_tablet_manager.cc:1950] T 906f207dee674a4288c8c60b5d3ff9f8 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.051560 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 155) took 9755 ms. Trace:
I20260320 15:27:44.051652 29678 rpcz_store.cc:276] 0320 15:27:34.295709 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.295904 (+ 195us) service_pool.cc:225] Handling call
0320 15:27:43.934439 (+9638535us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:43.934454 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:43.985583 (+ 51129us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.049849 (+ 64266us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.051516 (+ 1667us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9626357}
I20260320 15:27:44.051966 28733 tablet_bootstrap.cc:492] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.052188 29678 tablet_replica.cc:333] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.052204 28733 ts_tablet_manager.cc:1403] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.000s
I20260320 15:27:44.052398 29678 raft_consensus.cc:2243] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.052597 29678 raft_consensus.cc:2272] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.053164 28733 raft_consensus.cc:359] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.053352 28733 raft_consensus.cc:385] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.053568 29678 ts_tablet_manager.cc:1916] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.053597 28733 raft_consensus.cc:740] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.054093 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.054093 28733 consensus_queue.cc:260] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.054646 29643 ts_tablet_manager.cc:1916] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.057809 28733 ts_tablet_manager.cc:1434] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:44.058374 28733 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.060065 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.060608 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.061165 28733 tablet_bootstrap.cc:654] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.065938 30344 raft_consensus.cc:493] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.066187 30344 raft_consensus.cc:515] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.067417 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.068068 28382 leader_election.cc:343] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.071998 30344 leader_election.cc:290] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.073398 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec with delete_type TABLET_DATA_DELETED (replaced by tablet 6049ed1c2d84494893bfa20a9d563838) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.080786 28736 tablet_bootstrap.cc:654] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:44.067229 28524 consensus_peers.cc:597] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.082547 28494 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:44.056355 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 906f207dee674a4288c8c60b5d3ff9f8 (table default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]) successfully deleted
I20260320 15:27:44.084182 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7f0242a7968541f480367b45cda5d9cc (table default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]) successfully deleted
I20260320 15:27:44.084561 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet adb65832eefe4bd4b08d95aa83c7bd7b (table default.Table_76b6b411e37e4bdf807e460a44d52c20 [id=03747dfdb7724bbb84f1be9c9ed24922]) successfully deleted
W20260320 15:27:44.036593 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 121) took 2128 ms (client timeout 1000 ms). Trace:
I20260320 15:27:44.085493 30395 consensus_queue.cc:1048] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:44.085374 28904 rpcz_store.cc:269] 0320 15:27:41.908523 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.961280 (+ 52757us) service_pool.cc:225] Handling call
0320 15:27:44.036544 (+2075264us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.142996 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.143135 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.143143 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:43.143146 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:43.143294 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.143310 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.143315 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:43.143424 (+ 109us) tablet.cc:700] Row locks acquired
0320 15:27:43.143427 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:43.143466 (+ 39us) write_op.cc:270] Start()
0320 15:27:43.143503 (+ 37us) write_op.cc:276] Timestamp: P: 1774020463142209 usec, L: 0
0320 15:27:43.143506 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.143817 (+ 311us) log.cc:844] Serialized 901 byte log entry
0320 15:27:43.147361 (+ 3544us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.147499 (+ 138us) write_op.cc:301] APPLY: starting
0320 15:27:43.147519 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.147610 (+ 91us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.147649 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.147739 (+ 90us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.147786 (+ 47us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:43.147798 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:43.147890 (+ 92us) log.cc:844] Serialized 29 byte log entry
0320 15:27:43.147959 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.147985 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:43.147992 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.148077 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.148080 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:43.263655 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.263766 (+ 111us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.263773 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:43.263776 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:43.263892 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.263907 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.263911 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:43.263975 (+ 64us) tablet.cc:700] Row locks acquired
0320 15:27:43.263978 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:43.264010 (+ 32us) write_op.cc:270] Start()
0320 15:27:43.264070 (+ 60us) write_op.cc:276] Timestamp: P: 1774020463263990 usec, L: 0
0320 15:27:43.264074 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.264262 (+ 188us) log.cc:844] Serialized 356 byte log entry
0320 15:27:43.267796 (+ 3534us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.267917 (+ 121us) write_op.cc:301] APPLY: starting
0320 15:27:43.267936 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.267979 (+ 43us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.268005 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.268061 (+ 56us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.268099 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.268109 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:43.268188 (+ 79us) log.cc:844] Serialized 23 byte log entry
0320 15:27:43.268261 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.268280 (+ 19us) write_op.cc:454] Released schema lock
0320 15:27:43.268286 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.268354 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.268358 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":224431,"reported_tablets":190,"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":552,"apply.run_wall_time_us":560,"num_ops":2,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":942,"prepare.run_wall_time_us":943,"raft.queue_time_us":38,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":385,"replication_time_us":3820}],["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":426,"apply.run_wall_time_us":427,"num_ops":1,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":734,"prepare.run_wall_time_us":735,"raft.queue_time_us":384,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":252,"replication_time_us":3689}]]}
W20260320 15:27:44.073105 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 120) took 3326 ms (client timeout 1000 ms). Trace:
I20260320 15:27:44.087841 28733 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.087983 28733 ts_tablet_manager.cc:1403] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.001s
W20260320 15:27:44.087698 28889 rpcz_store.cc:269] 0320 15:27:40.746456 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.859213 (+112757us) service_pool.cc:225] Handling call
0320 15:27:44.073047 (+3213834us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:41.632998 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:41.640362 (+ 7364us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:41.640370 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:41.640372 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:41.640537 (+ 165us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:41.640552 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:41.640558 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:41.640678 (+ 120us) tablet.cc:700] Row locks acquired
0320 15:27:41.640680 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:41.640727 (+ 47us) write_op.cc:270] Start()
0320 15:27:41.640767 (+ 40us) write_op.cc:276] Timestamp: P: 1774020461640651 usec, L: 0
0320 15:27:41.640771 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:41.640994 (+ 223us) log.cc:844] Serialized 901 byte log entry
0320 15:27:41.746538 (+105544us) op_driver.cc:464] REPLICATION: finished
0320 15:27:41.784689 (+ 38151us) write_op.cc:301] APPLY: starting
0320 15:27:41.784708 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:41.784771 (+ 63us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:41.784843 (+ 72us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:41.785015 (+ 172us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:41.785053 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:41.785061 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:41.785186 (+ 125us) log.cc:844] Serialized 29 byte log entry
0320 15:27:41.785324 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:41.785346 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:41.785354 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":108370,"reported_tablets":190,"child_traces":[["op",{"apply.queue_time_us":38062,"apply.run_cpu_time_us":759,"apply.run_wall_time_us":759,"num_ops":2,"prepare.queue_time_us":83579,"prepare.run_cpu_time_us":1183,"prepare.run_wall_time_us":47185,"raft.queue_time_us":873,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":355,"replication_time_us":105706,"spinlock_wait_cycles":95104,"thread_start_us":122432,"threads_started":2}]]}
I20260320 15:27:44.088740 28733 raft_consensus.cc:359] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.088922 28733 raft_consensus.cc:385] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.088979 28733 raft_consensus.cc:740] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.088088 30256 raft_consensus.cc:515] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.096606 28733 consensus_queue.cc:260] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.098191 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.098409 28444 raft_consensus.cc:2468] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.099843 28905 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_213392d2bde147b2aae83c2b1e8eb8ea" } modify_external_catalogs: true
I20260320 15:27:44.100617 28905 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_213392d2bde147b2aae83c2b1e8eb8ea" } modify_external_catalogs: true
I20260320 15:27:44.101186 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.108707 28904 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2cf33331acf14700b31b2edd8666f285" } modify_external_catalogs: true
I20260320 15:27:44.108358 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.110157 28443 raft_consensus.cc:3060] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.110888 28736 tablet_bootstrap.cc:492] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.111147 28736 ts_tablet_manager.cc:1403] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.002s
I20260320 15:27:44.110005 28733 ts_tablet_manager.cc:1434] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.022s user 0.003s sys 0.000s
I20260320 15:27:44.113273 28733 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.114892 28733 tablet_bootstrap.cc:654] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.118484 30256 leader_election.cc:290] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:44.121191 28383 leader_election.cc:343] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.121384 28383 leader_election.cc:304] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.121843 30454 raft_consensus.cc:2749] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.122561 28736 raft_consensus.cc:359] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.122838 28736 raft_consensus.cc:385] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.122913 28736 raft_consensus.cc:740] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.123198 28736 consensus_queue.cc:260] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.123906 28736 ts_tablet_manager.cc:1434] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260320 15:27:44.124292 28736 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.126087 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Bootstrap replayed 1/1 log segments. Stats: ops{read=548 overwritten=1 applied=543 ignored=0} inserts{seen=505 ignored=0} mutations{seen=595 ignored=0} orphaned_commits=0. Pending: 4 replicates
I20260320 15:27:44.127172 29807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: Bootstrap complete.
I20260320 15:27:44.129904 28524 leader_election.cc:304] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:44.130486 30155 raft_consensus.cc:2804] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.130652 30155 raft_consensus.cc:493] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.130810 30155 raft_consensus.cc:3060] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.131011 30434 consensus_queue.cc:1048] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.134076 30155 raft_consensus.cc:515] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.134909 28443 raft_consensus.cc:2468] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:44.135385 30155 leader_election.cc:290] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.135638 28240 leader_election.cc:304] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.136116 30256 raft_consensus.cc:2804] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.136399 30256 raft_consensus.cc:697] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:44.136642 28734 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.136821 28734 ts_tablet_manager.cc:1403] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.184s user 0.004s sys 0.015s
I20260320 15:27:44.137725 30256 consensus_queue.cc:237] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.137923 28734 raft_consensus.cc:359] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.138108 28734 raft_consensus.cc:385] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.138166 28734 raft_consensus.cc:740] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.138409 28734 consensus_queue.cc:260] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.139303 28734 ts_tablet_manager.cc:1434] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260320 15:27:44.139623 28734 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.147876 28734 tablet_bootstrap.cc:654] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.150043 29643 ts_tablet_manager.cc:1929] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.150344 29643 log.cc:1199] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/71ec63fb080545de838803033f204fab
I20260320 15:27:44.150924 29643 ts_tablet_manager.cc:1950] T 71ec63fb080545de838803033f204fab P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.155761 30395 consensus_queue.cc:1048] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.156543 28904 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_2cf33331acf14700b31b2edd8666f285" } modify_external_catalogs: true
I20260320 15:27:44.146503 28736 tablet_bootstrap.cc:654] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.151659 30466 raft_consensus.cc:493] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.157716 30466 raft_consensus.cc:515] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.158633 30466 leader_election.cc:290] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:44.159520 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.159772 28441 raft_consensus.cc:2468] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:44.160216 28240 leader_election.cc:304] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.160753 30466 raft_consensus.cc:2804] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.160840 30466 raft_consensus.cc:493] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.160907 30466 raft_consensus.cc:3060] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.162669 28734 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.163813 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:44.163900 28734 ts_tablet_manager.cc:1403] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.006s
I20260320 15:27:44.164866 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:44.165050 28574 raft_consensus.cc:2468] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:44.165292 30454 raft_consensus.cc:493] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.165441 30454 raft_consensus.cc:515] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.166163 30454 leader_election.cc:290] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.166867 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.167431 28382 leader_election.cc:343] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.168673 30466 raft_consensus.cc:515] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.169363 29678 ts_tablet_manager.cc:1929] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.169514 29678 log.cc:1199] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f0a31d385b81456fb46ea52083874fcf
I20260320 15:27:44.169909 29678 ts_tablet_manager.cc:1950] T f0a31d385b81456fb46ea52083874fcf P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.170274 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 164) took 9772 ms. Trace:
I20260320 15:27:44.170367 29678 rpcz_store.cc:276] 0320 15:27:34.398110 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.398198 (+ 88us) service_pool.cc:225] Handling call
0320 15:27:44.052136 (+9653938us) ts_tablet_manager.cc:1151] Deleting tablet f0a31d385b81456fb46ea52083874fcf
0320 15:27:44.052146 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.126497 (+ 74351us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.169227 (+ 42730us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.170248 (+ 1021us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9644866}
I20260320 15:27:44.170586 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 155) took 9861 ms. Trace:
I20260320 15:27:44.170678 29643 rpcz_store.cc:276] 0320 15:27:34.316672 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.316917 (+ 245us) service_pool.cc:225] Handling call
0320 15:27:43.989424 (+9672507us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:43.989451 (+ 27us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.107471 (+118020us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.149869 (+ 42398us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.170561 (+ 20692us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9672281}
I20260320 15:27:44.170835 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 168) took 8580 ms. Trace:
I20260320 15:27:44.170905 29678 rpcz_store.cc:276] 0320 15:27:35.590864 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.590935 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:44.170693 (+8579758us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e17cea044541968356ae993be1e7
0320 15:27:44.170706 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.170818 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8579508}
I20260320 15:27:44.171249 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 169) took 8562 ms. Trace:
I20260320 15:27:44.171303 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.171321 29678 rpcz_store.cc:276] 0320 15:27:35.608667 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.608733 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:44.171146 (+8562413us) ts_tablet_manager.cc:1151] Deleting tablet 9aa2d48a54dd403c82d7f3a64db5aeab
0320 15:27:44.171155 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.171219 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8562122}
I20260320 15:27:44.171500 28446 raft_consensus.cc:3060] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.171998 30466 leader_election.cc:290] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:44.174657 28734 raft_consensus.cc:359] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.176867 28734 raft_consensus.cc:385] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.177830 28446 raft_consensus.cc:2468] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:44.178339 28240 leader_election.cc:304] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.177493 28734 raft_consensus.cc:740] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.178611 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.178726 30466 raft_consensus.cc:2804] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.178788 28446 raft_consensus.cc:3060] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.178831 30466 raft_consensus.cc:697] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:44.179096 30466 consensus_queue.cc:237] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.179917 28734 consensus_queue.cc:260] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.182360 28446 raft_consensus.cc:2468] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.184187 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 71ec63fb080545de838803033f204fab (table default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]) successfully deleted
I20260320 15:27:44.185593 28734 ts_tablet_manager.cc:1434] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.021s user 0.003s sys 0.001s
I20260320 15:27:44.188338 28734 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.198521 29640 ts_tablet_manager.cc:1929] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:44.198683 29640 log.cc:1199] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/07af28f257d745a1bb4173aad09045d9
I20260320 15:27:44.199007 29640 ts_tablet_manager.cc:1950] T 07af28f257d745a1bb4173aad09045d9 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.199321 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 142) took 10052 ms. Trace:
I20260320 15:27:44.199405 29640 rpcz_store.cc:276] 0320 15:27:34.146474 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.146573 (+ 99us) service_pool.cc:225] Handling call
0320 15:27:43.675033 (+9528460us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:43.675300 (+ 267us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.080059 (+404759us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.198373 (+118314us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.199303 (+ 930us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9527300}
I20260320 15:27:44.199936 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 07af28f257d745a1bb4173aad09045d9 (table default.Table_a861f30c768b4ee4988c1f3881eefa19 [id=1414f02e5e5b438596a1089b40ac0c21]) successfully deleted
W20260320 15:27:44.200451 28524 consensus_peers.cc:597] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f0242a7968541f480367b45cda5d9cc. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:44.200636 28524 consensus_peers.cc:597] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.201835 28733 tablet_bootstrap.cc:492] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.202106 28733 ts_tablet_manager.cc:1403] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.089s user 0.003s sys 0.013s
I20260320 15:27:44.203729 28733 raft_consensus.cc:359] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.204098 28733 raft_consensus.cc:385] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.204183 28733 raft_consensus.cc:740] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.204622 28733 consensus_queue.cc:260] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.205987 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:44.207187 28733 ts_tablet_manager.cc:1434] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:44.208091 29643 tablet_replica.cc:333] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.208616 29643 raft_consensus.cc:2243] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.208984 29643 raft_consensus.cc:2272] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.211614 29643 ts_tablet_manager.cc:1916] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.214238 28736 tablet_bootstrap.cc:492] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.214550 28736 ts_tablet_manager.cc:1403] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.090s user 0.007s sys 0.001s
I20260320 15:27:44.216450 28736 raft_consensus.cc:359] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.217562 28736 raft_consensus.cc:385] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.217475 28524 leader_election.cc:304] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.217645 28736 raft_consensus.cc:740] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.217960 30394 raft_consensus.cc:2804] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.217883 28736 consensus_queue.cc:260] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.218089 30394 raft_consensus.cc:697] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.218364 30394 consensus_queue.cc:237] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.218545 28736 ts_tablet_manager.cc:1434] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260320 15:27:44.219669 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 170) took 8602 ms. Trace:
I20260320 15:27:44.219794 29678 rpcz_store.cc:276] 0320 15:27:35.617046 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.617114 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:44.212684 (+8595570us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:44.212715 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.219639 (+ 6924us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8554189}
I20260320 15:27:44.220281 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 171) took 8600 ms. Trace:
I20260320 15:27:44.220366 29678 rpcz_store.cc:276] 0320 15:27:35.619718 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.619777 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:44.220167 (+8600390us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:44.220183 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.220264 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8600181}
I20260320 15:27:44.220758 29678 tablet_replica.cc:333] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.223353 28736 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.223956 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:44.224663 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:44.211123 29807 raft_consensus.cc:359] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Replica starting. Triggering 4 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:44.231956 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
W20260320 15:27:44.227475 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 60627us
I20260320 15:27:44.229732 28736 tablet_bootstrap.cc:654] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.230942 29678 raft_consensus.cc:2243] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.234951 29678 raft_consensus.cc:2272] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.235808 29678 ts_tablet_manager.cc:1916] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:44.236881 28239 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.234335 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:44.237522 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:44.238381 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:44.243979 28299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:44.243125 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet f0a31d385b81456fb46ea52083874fcf (table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]) successfully deleted
W20260320 15:27:44.244604 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8d92e17cea044541968356ae993be1e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d92e17cea044541968356ae993be1e7 already in progress: creating tablet
W20260320 15:27:44.244868 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because the tablet was not found. No further retry: Not found: Tablet not found: 9aa2d48a54dd403c82d7f3a64db5aeab
W20260320 15:27:44.245087 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet a2775196c8e04dcc80c7e089b57579fd because the tablet was not found. No further retry: Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
W20260320 15:27:44.245306 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet edb7b53bb7444a39b471218c6039542f because the tablet was not found. No further retry: Not found: Tablet not found: edb7b53bb7444a39b471218c6039542f
I20260320 15:27:44.246012 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:44.246176 28736 tablet_bootstrap.cc:492] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.246440 28736 ts_tablet_manager.cc:1403] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.000s
W20260320 15:27:44.250762 28870 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 71db28879e554f578b8a7b0e14419ab7 consensus state has no leader
I20260320 15:27:44.251132 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:44.251672 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:44.254938 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:44.255843 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:44.256582 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:44.256951 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:44.258255 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:44.258991 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:44.263729 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:44.264472 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:44.265280 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:44.265797 28903 catalog_manager.cc:5671] T 7fb1733e7a0e4774a5be2e06208bb495 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.266522 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:44.268455 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:44.269835 29807 raft_consensus.cc:740] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e8bb8819eb4cb4892e0cba1f758892, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.270851 29807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 543, Last appended: 4.547, Last appended by leader: 547, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:44.272581 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:44.279906 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:44.280331 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:44.281014 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:44.281226 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:44.282261 30492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:44.282620 30492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
W20260320 15:27:44.282956 28524 consensus_peers.cc:597] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ed29146bff74e86a366d3c6329176f4. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.285106 29807 sys_catalog.cc:565] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: configured and running, proceeding with master startup.
I20260320 15:27:44.285162 29721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 6 candidate_status { last_received { term: 5 index: 609 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:44.285287 29719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "336af1a126ca4ab583c03527b890239a" candidate_term: 6 candidate_status { last_received { term: 5 index: 609 } } ignore_live_leader: false dest_uuid: "07e8bb8819eb4cb4892e0cba1f758892" is_pre_election: true
I20260320 15:27:44.285928 29721 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 336af1a126ca4ab583c03527b890239a in term 4.
W20260320 15:27:44.286217 29721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53218 (request call id 0) took 9026 ms (client timeout 640 ms). Trace:
I20260320 15:27:44.286938 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
W20260320 15:27:44.286408 29721 rpcz_store.cc:269] 0320 15:27:35.259656 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.259747 (+ 91us) service_pool.cc:225] Handling call
0320 15:27:44.286116 (+9026369us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:44.294305 29678 ts_tablet_manager.cc:1929] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.294456 29678 log.cc:1199] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:44.294996 29678 ts_tablet_manager.cc:1950] T 27ccc693e7704650bd77e0c4ca11862e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.295620 28906 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 115) took 3374 ms. Trace:
I20260320 15:27:44.295754 28906 rpcz_store.cc:276] 0320 15:27:40.921279 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.018585 (+3097306us) service_pool.cc:225] Handling call
0320 15:27:44.295494 (+276909us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:44.296319 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d6cad22d93234701b9b022843b6fbaaa" } new_table_name: "default.Table_592f5890923d4f61aad0b52ea7be4aeb" modify_external_catalogs: true
I20260320 15:27:44.299819 28736 raft_consensus.cc:359] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.300024 28736 raft_consensus.cc:385] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.300079 28736 raft_consensus.cc:740] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.300285 28736 consensus_queue.cc:260] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.300938 28736 ts_tablet_manager.cc:1434] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.054s user 0.002s sys 0.000s
I20260320 15:27:44.308617 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.312403 28383 leader_election.cc:343] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.312687 28383 leader_election.cc:304] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.313333 30454 raft_consensus.cc:2749] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.315937 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d9b0f1936f54c699211c5ee9709f9ed" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:44.316141 28574 raft_consensus.cc:3060] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.318861 28574 raft_consensus.cc:2468] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
W20260320 15:27:44.325937 28524 leader_election.cc:343] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.326889 29640 tablet_replica.cc:333] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.327122 29640 raft_consensus.cc:2243] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.327486 29640 raft_consensus.cc:2272] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.328186 29640 ts_tablet_manager.cc:1916] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.330461 28734 tablet_bootstrap.cc:654] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.335928 28734 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.336093 28734 ts_tablet_manager.cc:1403] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.149s user 0.004s sys 0.004s
I20260320 15:27:44.336926 28734 raft_consensus.cc:359] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.337101 28734 raft_consensus.cc:385] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.337150 28734 raft_consensus.cc:740] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.337419 28734 consensus_queue.cc:260] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.339861 28733 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.340940 28734 ts_tablet_manager.cc:1434] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:44.341349 28733 tablet_bootstrap.cc:654] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.343931 28734 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.344712 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 91f393d2ca304b899f788add43da7e9a (DEFAULT_TABLE table=default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.344992 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 91f393d2ca304b899f788add43da7e9a (DEFAULT_TABLE table=default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.345386 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f393d2ca304b899f788add43da7e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.349431 28734 tablet_bootstrap.cc:654] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.350178 28736 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.351177 30395 raft_consensus.cc:493] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.351317 30395 raft_consensus.cc:515] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.352006 28736 tablet_bootstrap.cc:654] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.352113 30395 leader_election.cc:290] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:44.354269 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 172) took 8671 ms. Trace:
I20260320 15:27:44.354364 29678 rpcz_store.cc:276] 0320 15:27:35.682445 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.682861 (+ 416us) service_pool.cc:225] Handling call
0320 15:27:44.220693 (+8537832us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:44.220707 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.280892 (+ 60185us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.294152 (+ 13260us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.354245 (+ 60093us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54,"tablet-delete.queue_time_us":8537531}
I20260320 15:27:44.353196 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f393d2ca304b899f788add43da7e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.353297 29643 ts_tablet_manager.cc:1929] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:44.353792 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.355285 28441 raft_consensus.cc:2468] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.355772 28524 leader_election.cc:304] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.356040 29643 log.cc:1199] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ad4a30476e7146afae1e77dad078136e
I20260320 15:27:44.356426 29643 ts_tablet_manager.cc:1950] T ad4a30476e7146afae1e77dad078136e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.357164 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 156) took 10036 ms. Trace:
I20260320 15:27:44.357250 29643 rpcz_store.cc:276] 0320 15:27:34.322091 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.322359 (+ 268us) service_pool.cc:225] Handling call
0320 15:27:44.171168 (+9848809us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:44.171179 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.334729 (+163550us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.353145 (+ 18416us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.357143 (+ 3998us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9848511}
I20260320 15:27:44.357616 30395 raft_consensus.cc:2804] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.357708 30395 raft_consensus.cc:493] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.357766 30395 raft_consensus.cc:3060] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.357746 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ad4a30476e7146afae1e77dad078136e (table default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]) successfully deleted
I20260320 15:27:44.358101 29643 tablet_replica.cc:333] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.358284 29643 raft_consensus.cc:2243] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.358424 29643 raft_consensus.cc:2272] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.358947 29643 ts_tablet_manager.cc:1916] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:44.360065 28524 consensus_peers.cc:597] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.360349 30395 raft_consensus.cc:515] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.360917 30395 leader_election.cc:290] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:44.363510 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.363765 28441 raft_consensus.cc:3060] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.366428 28441 raft_consensus.cc:2468] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.366536 29643 ts_tablet_manager.cc:1929] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:44.366653 29643 log.cc:1199] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:44.367323 28524 leader_election.cc:304] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.367769 29678 tablet_replica.cc:333] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.368259 30355 raft_consensus.cc:2804] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.368398 30355 raft_consensus.cc:697] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.368673 30355 consensus_queue.cc:237] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.374014 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.377370 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:44.379820 29643 ts_tablet_manager.cc:1950] T 8dc27a4968ca4203b67dd3064fb03868 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.379812 29678 raft_consensus.cc:2243] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.380852 29678 raft_consensus.cc:2272] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:44.286206 29719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53218 (request call id 1) took 8280 ms (client timeout 640 ms). Trace:
I20260320 15:27:44.380884 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 157) took 10057 ms. Trace:
W20260320 15:27:44.381240 29719 rpcz_store.cc:269] 0320 15:27:36.005985 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.006083 (+ 98us) service_pool.cc:225] Handling call
0320 15:27:44.286124 (+8280041us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:44.381279 29643 rpcz_store.cc:276] 0320 15:27:34.324782 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.325368 (+ 586us) service_pool.cc:225] Handling call
0320 15:27:44.358051 (+10032683us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:44.358061 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.363058 (+ 4997us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.366383 (+ 3325us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.380833 (+ 14450us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10032467}
I20260320 15:27:44.381839 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8dc27a4968ca4203b67dd3064fb03868 (table default.Table_4df55a5738d24d4292a69c7e5550865b [id=d903f8d305be49e6b984cb233844c1d2]) successfully deleted
I20260320 15:27:44.382351 29643 tablet_replica.cc:333] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.383208 29643 raft_consensus.cc:2243] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.383735 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 27ccc693e7704650bd77e0c4ca11862e (table default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]) successfully deleted
W20260320 15:27:44.384133 28524 consensus_peers.cc:597] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:44.384781 28524 leader_election.cc:343] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:44.385807 28524 leader_election.cc:343] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.383152 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 91f393d2ca304b899f788add43da7e9a (DEFAULT_TABLE table=default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.390626 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91f393d2ca304b899f788add43da7e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.395279 29643 raft_consensus.cc:2272] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.398926 29736 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 4 FOLLOWER]: Advancing to term 5
I20260320 15:27:44.399405 29678 ts_tablet_manager.cc:1916] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.400887 29643 ts_tablet_manager.cc:1916] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.401248 28733 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.401437 28733 ts_tablet_manager.cc:1403] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.062s user 0.005s sys 0.003s
I20260320 15:27:44.401690 30454 raft_consensus.cc:493] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.401916 30454 raft_consensus.cc:515] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.402233 28733 raft_consensus.cc:359] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.402407 28733 raft_consensus.cc:385] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.402451 28733 raft_consensus.cc:740] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.402645 30454 leader_election.cc:290] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.402869 28733 consensus_queue.cc:260] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.403394 28733 ts_tablet_manager.cc:1434] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:44.403263 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:44.403721 28733 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:44.403988 28382 leader_election.cc:343] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.405670 28733 tablet_bootstrap.cc:654] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.407318 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.408854 28383 leader_election.cc:343] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.409010 28383 leader_election.cc:304] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.409281 30454 raft_consensus.cc:2749] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:44.409955 28524 consensus_peers.cc:597] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 906f207dee674a4288c8c60b5d3ff9f8. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.413497 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 6049ed1c2d84494893bfa20a9d563838 (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.413975 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6049ed1c2d84494893bfa20a9d563838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.416391 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 6049ed1c2d84494893bfa20a9d563838 (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.417434 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6049ed1c2d84494893bfa20a9d563838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.416031 28736 tablet_bootstrap.cc:492] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.423893 28736 ts_tablet_manager.cc:1403] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.001s
I20260320 15:27:44.426033 28736 raft_consensus.cc:359] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.426362 29640 ts_tablet_manager.cc:1929] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.426522 29640 log.cc:1199] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ecaa08ba34514c5e99a9e2d4ba9268a9
I20260320 15:27:44.426964 29640 ts_tablet_manager.cc:1950] T ecaa08ba34514c5e99a9e2d4ba9268a9 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.427382 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 143) took 10280 ms. Trace:
I20260320 15:27:44.427642 28736 raft_consensus.cc:385] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.427837 28736 raft_consensus.cc:740] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.429499 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 6049ed1c2d84494893bfa20a9d563838 (DEFAULT_TABLE table=default.Table_7744ce92ac4e49be9f107f055594b6b2 [id=4ad8d840f81149aeb6f595ee2e2ddc55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:44.430065 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6049ed1c2d84494893bfa20a9d563838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:44.430017 30355 consensus_queue.cc:1048] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:44.431777 30497 log.cc:927] Time spent T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260320 15:27:44.438585 28734 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.439147 30454 raft_consensus.cc:493] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.439395 30454 raft_consensus.cc:515] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.439648 28734 ts_tablet_manager.cc:1403] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.003s
I20260320 15:27:44.440222 30454 leader_election.cc:290] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.441139 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.441814 28382 leader_election.cc:343] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.442257 29643 ts_tablet_manager.cc:1929] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.442430 29643 log.cc:1199] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/3ccbf4f32636463f8e1e005a629b92cc
I20260320 15:27:44.442549 28734 raft_consensus.cc:359] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.442726 28734 raft_consensus.cc:385] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.442799 28734 raft_consensus.cc:740] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.443866 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.443893 28734 consensus_queue.cc:260] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:44.445654 28239 consensus_peers.cc:597] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:44.446087 28383 leader_election.cc:343] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.446267 28383 leader_election.cc:304] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.446810 30456 raft_consensus.cc:2749] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:44.447479 28524 consensus_peers.cc:597] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71ec63fb080545de838803033f204fab. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.427500 29640 rpcz_store.cc:276] 0320 15:27:34.147065 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.147115 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:44.199718 (+10052603us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:44.199730 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.338038 (+138308us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.426205 (+ 88167us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.427359 (+ 1154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10051873}
I20260320 15:27:44.452054 29640 tablet_replica.cc:333] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.452344 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ecaa08ba34514c5e99a9e2d4ba9268a9 (table default.Table_30015eb07aba409bb03a37140eea20d6 [id=83739ddc977f4089b85ad8e6ea64df8b]) successfully deleted
I20260320 15:27:44.452421 29640 raft_consensus.cc:2243] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.452756 29640 raft_consensus.cc:2272] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.452992 28733 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.453116 28733 ts_tablet_manager.cc:1403] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.000s
I20260320 15:27:44.453359 29640 ts_tablet_manager.cc:1916] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.453785 28733 raft_consensus.cc:359] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.453951 28733 raft_consensus.cc:385] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.454000 28733 raft_consensus.cc:740] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.454186 28733 consensus_queue.cc:260] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.454627 28733 ts_tablet_manager.cc:1434] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260320 15:27:44.458415 28733 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.460479 28733 tablet_bootstrap.cc:654] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.463218 28733 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.463371 28733 ts_tablet_manager.cc:1403] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260320 15:27:44.464123 28733 raft_consensus.cc:359] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.464298 28733 raft_consensus.cc:385] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.464351 28733 raft_consensus.cc:740] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.464560 28733 consensus_queue.cc:260] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.465047 28733 ts_tablet_manager.cc:1434] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:44.465358 28733 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.467283 28733 tablet_bootstrap.cc:654] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.428767 28736 consensus_queue.cc:260] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.469563 28300 raft_consensus.cc:3060] T a3bd9c79d2d74e878647f97e66a9bfca P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.469944 28736 ts_tablet_manager.cc:1434] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.045s user 0.002s sys 0.000s
I20260320 15:27:44.470293 28736 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.471808 29643 ts_tablet_manager.cc:1950] T 3ccbf4f32636463f8e1e005a629b92cc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.472163 28736 tablet_bootstrap.cc:654] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.472936 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 158) took 10146 ms. Trace:
I20260320 15:27:44.473038 29643 rpcz_store.cc:276] 0320 15:27:34.327476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.327846 (+ 370us) service_pool.cc:225] Handling call
0320 15:27:44.382277 (+10054431us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:44.382288 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.405142 (+ 22854us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.442086 (+ 36944us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.472890 (+ 30804us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10054215}
I20260320 15:27:44.473587 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 3ccbf4f32636463f8e1e005a629b92cc (table default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]) successfully deleted
I20260320 15:27:44.474773 28734 ts_tablet_manager.cc:1434] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20260320 15:27:44.475181 28734 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.476984 28734 tablet_bootstrap.cc:654] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:44.480225 28240 consensus_peers.cc:597] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.481148 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:44.483990 30395 consensus_queue.cc:1048] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.484644 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:44.484949 29643 tablet_replica.cc:333] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.485229 29643 raft_consensus.cc:2243] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.485556 29643 pending_rounds.cc:70] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Trying to abort 1 pending ops.
I20260320 15:27:44.486565 29643 pending_rounds.cc:77] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7266387820365443072 op_type: NO_OP noop_request { }
I20260320 15:27:44.487287 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:44.487579 29643 raft_consensus.cc:2889] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:44.487761 29643 raft_consensus.cc:2272] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.487852 30395 raft_consensus.cc:493] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.487986 30395 raft_consensus.cc:515] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.488690 30395 leader_election.cc:290] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.488866 29643 ts_tablet_manager.cc:1916] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.487697 30355 consensus_queue.cc:1048] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260320 15:27:44.489513 29640 ts_tablet_manager.cc:1929] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.490914 28299 raft_consensus.cc:3060] T 6197f6a1e1cf4007a481be3d7ccfd425 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.490790 28889 catalog_manager.cc:5671] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.491578 29678 ts_tablet_manager.cc:1929] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.491703 29678 log.cc:1199] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/07ff84dedc074dabb4e8e2d8460066e7
I20260320 15:27:44.492081 29678 ts_tablet_manager.cc:1950] T 07ff84dedc074dabb4e8e2d8460066e7 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.492416 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 175) took 8809 ms. Trace:
I20260320 15:27:44.492522 29678 rpcz_store.cc:276] 0320 15:27:35.682715 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.682966 (+ 251us) service_pool.cc:225] Handling call
0320 15:27:44.367708 (+8684742us) ts_tablet_manager.cc:1151] Deleting tablet 07ff84dedc074dabb4e8e2d8460066e7
0320 15:27:44.367722 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.414937 (+ 47215us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.491435 (+ 76498us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.492378 (+ 943us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8684526}
I20260320 15:27:44.492455 28889 catalog_manager.cc:5671] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.492857 29678 tablet_replica.cc:333] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.493063 29678 raft_consensus.cc:2243] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.493247 29678 raft_consensus.cc:2272] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.493527 28889 catalog_manager.cc:5671] T e5479578a7cd470a9a4851d252a9e93e P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.493856 29678 ts_tablet_manager.cc:1916] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.494179 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:44.495955 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]) successfully deleted
I20260320 15:27:44.496464 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:44.496870 28889 catalog_manager.cc:5671] T a3bd9c79d2d74e878647f97e66a9bfca P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
W20260320 15:27:44.489611 28524 leader_election.cc:343] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ad4a30476e7146afae1e77dad078136e
W20260320 15:27:44.502827 28524 leader_election.cc:343] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: ad4a30476e7146afae1e77dad078136e
I20260320 15:27:44.502976 28524 leader_election.cc:304] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:44.503275 30355 raft_consensus.cc:2749] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.503465 30394 consensus_queue.cc:1048] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.507493 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:44.508412 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:44.509881 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:44.511257 28733 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.511600 28733 ts_tablet_manager.cc:1403] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.046s user 0.004s sys 0.001s
W20260320 15:27:44.512341 28524 consensus_peers.cc:597] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f0242a7968541f480367b45cda5d9cc. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.512780 28733 raft_consensus.cc:359] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.513059 28733 raft_consensus.cc:385] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.513175 28733 raft_consensus.cc:740] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.513516 28733 consensus_queue.cc:260] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.515730 28733 ts_tablet_manager.cc:1434] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:44.515872 28734 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.515985 29640 log.cc:1199] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/bdd8f5ccbd35418684fd27459c7a6b9f
I20260320 15:27:44.516047 28734 ts_tablet_manager.cc:1403] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.000s
I20260320 15:27:44.516281 28733 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.516876 28734 raft_consensus.cc:359] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.517079 28734 raft_consensus.cc:385] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.517164 28734 raft_consensus.cc:740] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.517408 28734 consensus_queue.cc:260] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.517931 28734 ts_tablet_manager.cc:1434] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:44.518245 28734 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.518348 28733 tablet_bootstrap.cc:654] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.520293 30395 consensus_queue.cc:1048] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.521040 30456 raft_consensus.cc:493] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.521198 30456 raft_consensus.cc:515] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.521871 30456 leader_election.cc:290] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.522010 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:44.522326 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:44.522576 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:44.522823 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:44.522966 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
W20260320 15:27:44.523815 28382 leader_election.cc:343] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.524327 29640 ts_tablet_manager.cc:1950] T bdd8f5ccbd35418684fd27459c7a6b9f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.526170 30395 consensus_queue.cc:1048] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.526031 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.527665 28383 leader_election.cc:343] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.527807 28383 leader_election.cc:304] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.528021 30344 raft_consensus.cc:2749] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.533351 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 144) took 10381 ms. Trace:
I20260320 15:27:44.533560 29640 rpcz_store.cc:276] 0320 15:27:34.151681 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.167914 (+ 16233us) service_pool.cc:225] Handling call
0320 15:27:44.451991 (+10284077us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:44.452004 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.458949 (+ 6945us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.489401 (+ 30452us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.533322 (+ 43921us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10270855}
I20260320 15:27:44.534492 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet bdd8f5ccbd35418684fd27459c7a6b9f (table default.Table_a5952001c570473c902abb9d529c60f1 [id=0eb95741e0d74c2fb248cdef0ae91c08]) successfully deleted
I20260320 15:27:44.535336 29640 tablet_replica.cc:333] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.535738 29640 raft_consensus.cc:2243] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.536015 29640 raft_consensus.cc:2272] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.536813 29640 ts_tablet_manager.cc:1916] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.538926 29643 ts_tablet_manager.cc:1929] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260320 15:27:44.539110 30344 raft_consensus.cc:493] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.539288 30344 raft_consensus.cc:515] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.523488 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:44.540025 30344 leader_election.cc:290] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.540742 28889 catalog_manager.cc:5671] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.541177 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:44.541987 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:44.542214 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:44.543026 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:44.543319 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:44.539124 29643 log.cc:1199] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/aafd1e123cd043e7828140d2443e047f
I20260320 15:27:44.546216 29643 ts_tablet_manager.cc:1950] T aafd1e123cd043e7828140d2443e047f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.547178 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 159) took 10217 ms. Trace:
I20260320 15:27:44.547297 29643 rpcz_store.cc:276] 0320 15:27:34.330084 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.332868 (+ 2784us) service_pool.cc:225] Handling call
0320 15:27:44.484879 (+10152011us) ts_tablet_manager.cc:1151] Deleting tablet aafd1e123cd043e7828140d2443e047f
0320 15:27:44.484889 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.503723 (+ 18834us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.538715 (+ 34992us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.547116 (+ 8401us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10151795}
I20260320 15:27:44.547734 29643 tablet_replica.cc:333] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.547888 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet aafd1e123cd043e7828140d2443e047f (table default.Table_134b3b5c79e344f2bf710b12615479be [id=5186e4c77f9f4f59814887534793f0f0]) successfully deleted
I20260320 15:27:44.547960 29643 raft_consensus.cc:2243] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.548187 29643 raft_consensus.cc:2272] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.548835 29643 ts_tablet_manager.cc:1916] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.549018 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.549805 28383 leader_election.cc:343] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.544087 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.550983 28382 leader_election.cc:343] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.551123 28382 leader_election.cc:304] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.551644 30344 raft_consensus.cc:2749] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.553098 28736 tablet_bootstrap.cc:492] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.556157 28736 ts_tablet_manager.cc:1403] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.086s user 0.000s sys 0.004s
I20260320 15:27:44.557061 28736 raft_consensus.cc:359] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.562438 28736 raft_consensus.cc:385] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.562657 28736 raft_consensus.cc:740] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.562988 28736 consensus_queue.cc:260] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.564577 30344 raft_consensus.cc:493] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.564757 30344 raft_consensus.cc:515] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.565513 30344 leader_election.cc:290] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.566588 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.567221 28382 leader_election.cc:343] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.572019 28736 ts_tablet_manager.cc:1434] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260320 15:27:44.566252 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.580380 28524 consensus_peers.cc:597] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 906f207dee674a4288c8c60b5d3ff9f8. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.585397 28734 tablet_bootstrap.cc:654] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.586802 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:44.587620 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:44.588209 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:44.588420 28736 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:44.591354 28383 leader_election.cc:343] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.591558 28383 leader_election.cc:304] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.591820 30344 raft_consensus.cc:2749] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:44.597407 28524 consensus_peers.cc:597] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.598533 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:44.599102 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:44.599933 28889 catalog_manager.cc:5671] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.601105 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:44.602080 28889 catalog_manager.cc:5671] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:44.602605 30155 consensus_queue.cc:1048] T 6197f6a1e1cf4007a481be3d7ccfd425 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.605844 29640 ts_tablet_manager.cc:1929] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.607860 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:44.616873 28736 tablet_bootstrap.cc:654] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.622418 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:44.628319 28889 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:44.633864 29643 ts_tablet_manager.cc:1929] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.634014 29643 log.cc:1199] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f0a31d385b81456fb46ea52083874fcf
I20260320 15:27:44.634549 29643 ts_tablet_manager.cc:1950] T f0a31d385b81456fb46ea52083874fcf P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.647018 29640 log.cc:1199] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:44.632820 28733 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.647505 29640 ts_tablet_manager.cc:1950] T 20a7164db1e0447fb424d8e46b747f52 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.648416 28734 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.648597 28734 ts_tablet_manager.cc:1403] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.130s user 0.002s sys 0.002s
I20260320 15:27:44.648871 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 145) took 10464 ms. Trace:
I20260320 15:27:44.648967 29640 rpcz_store.cc:276] 0320 15:27:34.184928 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.184996 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:44.534870 (+10349874us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:44.535278 (+ 408us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.554114 (+ 18836us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.605719 (+ 51605us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.648847 (+ 43128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10337168}
I20260320 15:27:44.649683 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 20a7164db1e0447fb424d8e46b747f52 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]) successfully deleted
I20260320 15:27:44.649674 28734 raft_consensus.cc:359] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.649899 28734 raft_consensus.cc:385] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.647991 28733 ts_tablet_manager.cc:1403] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.132s user 0.046s sys 0.053s
I20260320 15:27:44.649988 28734 raft_consensus.cc:740] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.650300 28734 consensus_queue.cc:260] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.650722 28733 raft_consensus.cc:359] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.650895 28733 raft_consensus.cc:385] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.650956 28733 raft_consensus.cc:740] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.651005 28734 ts_tablet_manager.cc:1434] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:44.651177 28733 consensus_queue.cc:260] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:44.651386 28524 consensus_peers.cc:597] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.651719 28733 ts_tablet_manager.cc:1434] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:44.652042 28733 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.652076 28635 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
I20260320 15:27:44.653879 28733 tablet_bootstrap.cc:654] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.654201 29678 ts_tablet_manager.cc:1929] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.654366 29678 log.cc:1199] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6771bd6a3f0e47d99a0191eae25cb2d2
I20260320 15:27:44.654773 29678 ts_tablet_manager.cc:1950] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.657689 29640 tablet_replica.cc:333] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.657917 29640 raft_consensus.cc:2243] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.658279 29640 raft_consensus.cc:2272] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.658915 29640 ts_tablet_manager.cc:1916] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.664060 30344 consensus_queue.cc:1048] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.666972 30395 raft_consensus.cc:493] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.667135 30395 raft_consensus.cc:515] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.668599 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 163) took 10278 ms. Trace:
I20260320 15:27:44.668761 29643 rpcz_store.cc:276] 0320 15:27:34.389802 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.389866 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:44.547650 (+10157784us) ts_tablet_manager.cc:1151] Deleting tablet f0a31d385b81456fb46ea52083874fcf
0320 15:27:44.547661 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.604179 (+ 56518us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.633698 (+ 29519us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.668530 (+ 34832us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10157503}
I20260320 15:27:44.669200 30395 leader_election.cc:290] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.669256 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet f0a31d385b81456fb46ea52083874fcf (table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]) successfully deleted
I20260320 15:27:44.669912 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.670171 28441 raft_consensus.cc:2468] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.670680 28524 leader_election.cc:304] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.671343 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.671645 28297 raft_consensus.cc:2468] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.673987 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:44.674680 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:44.674743 30344 raft_consensus.cc:493] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.674993 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:44.674945 30344 raft_consensus.cc:515] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.675763 30344 leader_election.cc:290] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.677130 30395 raft_consensus.cc:2804] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.677271 30395 raft_consensus.cc:493] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.677407 30395 raft_consensus.cc:3060] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.676704 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.678434 28382 leader_election.cc:343] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.680263 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:44.680899 28383 leader_election.cc:343] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.681059 28383 leader_election.cc:304] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:44.681627 30344 raft_consensus.cc:2749] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:44.683035 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:44.683228 28736 tablet_bootstrap.cc:492] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.683507 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:44.683516 28736 ts_tablet_manager.cc:1403] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.109s user 0.004s sys 0.000s
I20260320 15:27:44.683717 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:44.683884 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:44.684432 28733 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.684414 28736 raft_consensus.cc:359] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.684612 28733 ts_tablet_manager.cc:1403] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.000s
I20260320 15:27:44.685386 28733 raft_consensus.cc:359] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.685588 28733 raft_consensus.cc:385] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.685652 28733 raft_consensus.cc:740] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.684620 28736 raft_consensus.cc:385] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.685820 28736 raft_consensus.cc:740] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.685935 28733 consensus_queue.cc:260] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.686064 28736 consensus_queue.cc:260] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.686471 28733 ts_tablet_manager.cc:1434] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:44.691614 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:44.691921 28733 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.692054 29640 ts_tablet_manager.cc:1929] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.692443 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:44.692619 29640 log.cc:1199] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:44.693534 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:44.693727 29640 ts_tablet_manager.cc:1950] T ef09e4afdd1f4fce9caec4dcafa2c50c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.693954 28733 tablet_bootstrap.cc:654] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.694375 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:44.694494 28736 ts_tablet_manager.cc:1434] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.011s user 0.000s sys 0.002s
I20260320 15:27:44.695015 30395 raft_consensus.cc:515] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.655632 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 182) took 8866 ms. Trace:
I20260320 15:27:44.695439 29678 rpcz_store.cc:276] 0320 15:27:35.788940 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.788994 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:44.492790 (+8703796us) ts_tablet_manager.cc:1151] Deleting tablet 6771bd6a3f0e47d99a0191eae25cb2d2
0320 15:27:44.492799 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.623507 (+130708us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.653973 (+ 30466us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.655594 (+ 1621us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8703618}
I20260320 15:27:44.695138 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 146) took 10507 ms. Trace:
I20260320 15:27:44.695931 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 186) took 7701 ms. Trace:
I20260320 15:27:44.696026 29678 rpcz_store.cc:276] 0320 15:27:36.994646 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.994691 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:44.695783 (+7701092us) ts_tablet_manager.cc:1151] Deleting tablet 7f94140a5e8b4a06890bca86456d64a6
0320 15:27:44.695793 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.695895 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7700872}
I20260320 15:27:44.695853 29640 rpcz_store.cc:276] 0320 15:27:34.188029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.188081 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:44.657627 (+10469546us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:44.657641 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.687479 (+ 29838us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.690879 (+ 3400us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.695117 (+ 4238us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10456198}
I20260320 15:27:44.696346 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 190) took 7670 ms. Trace:
I20260320 15:27:44.696431 29678 rpcz_store.cc:276] 0320 15:27:37.026404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.026482 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:44.696228 (+7669746us) ts_tablet_manager.cc:1151] Deleting tablet ae3422cf33324ab89a2b44e27cae0eb2
0320 15:27:44.696237 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.696318 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7669496}
I20260320 15:27:44.696369 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:44.696698 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 193) took 7614 ms. Trace:
I20260320 15:27:44.696785 29678 rpcz_store.cc:276] 0320 15:27:37.081829 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.081895 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:44.696618 (+7614723us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:44.696625 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.696668 (+ 43us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7614515}
I20260320 15:27:44.697005 29640 tablet_replica.cc:333] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.697034 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 194) took 7614 ms. Trace:
I20260320 15:27:44.697117 29678 rpcz_store.cc:276] 0320 15:27:37.082281 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.082326 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:44.696958 (+7614632us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:44.696966 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.697005 (+ 39us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7614465}
I20260320 15:27:44.697376 29678 tablet_replica.cc:333] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.697750 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:44.697926 29640 raft_consensus.cc:2243] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.698192 29640 raft_consensus.cc:2272] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.697955 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (table default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]) successfully deleted
I20260320 15:27:44.698479 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ef09e4afdd1f4fce9caec4dcafa2c50c (table default.Table_864c3add59ac47ec8bbbcb4f4796beee [id=7ce284446e8c4dbba1a1f6eaa1708bf2]) successfully deleted
W20260320 15:27:44.698689 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
I20260320 15:27:44.699213 30395 leader_election.cc:290] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.699746 28736 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.700350 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:44.701007 29640 ts_tablet_manager.cc:1916] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.701324 28736 tablet_bootstrap.cc:654] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.702553 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.702767 28444 raft_consensus.cc:3060] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.705720 28444 raft_consensus.cc:2468] T 7e13c75a376c47c9a0083173865ce4fa P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.706292 28524 leader_election.cc:304] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.706816 30395 raft_consensus.cc:2804] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.707054 30395 raft_consensus.cc:697] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.707049 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:44.707338 30395 consensus_queue.cc:237] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.707715 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:44.708743 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:44.713733 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_d6cad22d93234701b9b022843b6fbaaa [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 2 got 0
I20260320 15:27:44.715005 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:44.715871 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:44.716398 28734 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.717170 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:44.717851 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:44.718242 28734 tablet_bootstrap.cc:654] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.718602 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:44.719189 29643 tablet_replica.cc:333] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.719460 29643 raft_consensus.cc:2243] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.719717 29643 raft_consensus.cc:2272] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.720767 29643 ts_tablet_manager.cc:1916] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.720985 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:44.724879 29640 ts_tablet_manager.cc:1929] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.725617 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:44.726936 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:44.728230 28892 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:44.728613 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:44.729310 28905 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 116) took 3665 ms. Trace:
I20260320 15:27:44.729429 28905 rpcz_store.cc:276] 0320 15:27:41.063348 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.090029 (+3026681us) service_pool.cc:225] Handling call
0320 15:27:44.100911 (+ 10882us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:44.101103 (+ 192us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:44.101196 (+ 93us) catalog_manager.cc:2824] Locking tablets
0320 15:27:44.101716 (+ 520us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:44.129278 (+ 27562us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:44.167460 (+ 38182us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:44.728555 (+561095us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:44.729214 (+ 659us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.102788 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.102985 (+ 197us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.102995 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:44.102999 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:44.103148 (+ 149us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.103170 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.103177 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.103337 (+ 160us) tablet.cc:700] Row locks acquired
0320 15:27:44.103341 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.103388 (+ 47us) write_op.cc:270] Start()
0320 15:27:44.103435 (+ 47us) write_op.cc:276] Timestamp: P: 1774020464103340 usec, L: 0
0320 15:27:44.103440 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.103722 (+ 282us) log.cc:844] Serialized 725 byte log entry
0320 15:27:44.109091 (+ 5369us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.109334 (+ 243us) write_op.cc:301] APPLY: starting
0320 15:27:44.109357 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.109412 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.128783 (+ 19371us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.128890 (+ 107us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.128946 (+ 56us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:44.128952 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:44.129056 (+ 104us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.129136 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.129163 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:44.129171 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.129255 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.129259 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":181,"apply.run_cpu_time_us":613,"apply.run_wall_time_us":19908,"num_ops":2,"prepare.queue_time_us":74,"prepare.run_cpu_time_us":1056,"prepare.run_wall_time_us":1058,"raft.queue_time_us":560,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":328,"replication_time_us":5607}]]}
I20260320 15:27:44.730371 28905 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8b5b70bee34f4caabd4a276b20f57793" } modify_external_catalogs: true
I20260320 15:27:44.730571 28905 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8b5b70bee34f4caabd4a276b20f57793" } modify_external_catalogs: true
I20260320 15:27:44.732124 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet e2243e6811744b7cba99a487f8e8bb83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.732609 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:44.733105 28904 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 117) took 3079 ms. Trace:
I20260320 15:27:44.733186 28904 rpcz_store.cc:276] 0320 15:27:41.653563 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.090662 (+2437099us) service_pool.cc:225] Handling call
0320 15:27:44.156736 (+ 66074us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:44.167616 (+ 10880us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:44.167699 (+ 83us) catalog_manager.cc:2824] Locking tablets
0320 15:27:44.300056 (+132357us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:44.308929 (+ 8873us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:44.309022 (+ 93us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:44.732553 (+423531us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:44.733052 (+ 499us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.301371 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.301531 (+ 160us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.301540 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:44.301544 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:44.301711 (+ 167us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.301725 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.301730 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.301853 (+ 123us) tablet.cc:700] Row locks acquired
0320 15:27:44.301857 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.301906 (+ 49us) write_op.cc:270] Start()
0320 15:27:44.301951 (+ 45us) write_op.cc:276] Timestamp: P: 1774020464301884 usec, L: 0
0320 15:27:44.301956 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.302138 (+ 182us) log.cc:844] Serialized 765 byte log entry
0320 15:27:44.308190 (+ 6052us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.308340 (+ 150us) write_op.cc:301] APPLY: starting
0320 15:27:44.308358 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.308410 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.308444 (+ 34us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.308528 (+ 84us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.308565 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:44.308573 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.308653 (+ 80us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.308708 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.308727 (+ 19us) write_op.cc:454] Released schema lock
0320 15:27:44.308733 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.308814 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.308818 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":466,"apply.run_wall_time_us":466,"num_ops":2,"prepare.queue_time_us":302,"prepare.run_cpu_time_us":928,"prepare.run_wall_time_us":928,"replication_time_us":6201,"spinlock_wait_cycles":165120}]]}
I20260320 15:27:44.733958 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_36123e5c9fed4e51bfab33f47ed1d82b" } new_table_name: "default.Table_ab12be810262486ca2ec3234763b510b" modify_external_catalogs: true
I20260320 15:27:44.737017 29640 log.cc:1199] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7feb8477ebf841a19977a434f365f537
I20260320 15:27:44.737342 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a3d5e4ff8f42409482af49c636be0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.737442 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a3d5e4ff8f42409482af49c636be0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.737509 29640 ts_tablet_manager.cc:1950] T 7feb8477ebf841a19977a434f365f537 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.727873 28733 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.737993 28733 ts_tablet_manager.cc:1403] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.010s
I20260320 15:27:44.738812 28733 raft_consensus.cc:359] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.739146 28890 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b] alter complete (version 1)
I20260320 15:27:44.744623 28900 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_84e513e1494a4b71ab7f8c61376aaeae [id=75466621ee1941378a891a480a3df191] alter complete (version 1)
I20260320 15:27:44.750555 29678 raft_consensus.cc:2243] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.750978 29678 raft_consensus.cc:2272] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.751626 29678 ts_tablet_manager.cc:1916] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.757655 28736 tablet_bootstrap.cc:492] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.757804 28736 ts_tablet_manager.cc:1403] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.002s
I20260320 15:27:44.758529 28736 raft_consensus.cc:359] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.758739 28736 raft_consensus.cc:385] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.758805 28736 raft_consensus.cc:740] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.759024 28736 consensus_queue.cc:260] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.759701 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.760428 28736 ts_tablet_manager.cc:1434] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:44.760710 28736 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.763705 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.763839 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.763747 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.764210 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet adb65832eefe4bd4b08d95aa83c7bd7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.764415 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.764487 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.764679 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dc27a4968ca4203b67dd3064fb03868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.767663 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:44.769238 28734 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.769411 28734 ts_tablet_manager.cc:1403] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.002s
W20260320 15:27:44.769951 28900 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 106) took 3660 ms (client timeout 1000 ms). Trace:
W20260320 15:27:44.770113 28900 rpcz_store.cc:269] 0320 15:27:41.109005 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.437134 (+328129us) service_pool.cc:225] Handling call
0320 15:27:44.063720 (+2626586us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.063728 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:44.769872 (+706144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.050755 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.050906 (+ 151us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.050915 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:44.050917 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.051050 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.051071 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.051079 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.051156 (+ 77us) tablet.cc:700] Row locks acquired
0320 15:27:44.051159 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.051208 (+ 49us) write_op.cc:270] Start()
0320 15:27:44.051249 (+ 41us) write_op.cc:276] Timestamp: P: 1774020464051163 usec, L: 0
0320 15:27:44.051343 (+ 94us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.051634 (+ 291us) log.cc:844] Serialized 356 byte log entry
0320 15:27:44.056000 (+ 4366us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.058491 (+ 2491us) write_op.cc:301] APPLY: starting
0320 15:27:44.058513 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.058556 (+ 43us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.058628 (+ 72us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.058704 (+ 76us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.058738 (+ 34us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.058746 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.058865 (+ 119us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.063343 (+ 4478us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.063377 (+ 34us) write_op.cc:454] Released schema lock
0320 15:27:44.063389 (+ 12us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":36476,"reported_tablets":195,"child_traces":[["op",{"apply.queue_time_us":2400,"apply.run_cpu_time_us":653,"apply.run_wall_time_us":4992,"num_ops":1,"prepare.queue_time_us":11562,"prepare.run_cpu_time_us":1014,"prepare.run_wall_time_us":1014,"raft.queue_time_us":785,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":279,"replication_time_us":4621}]]}
I20260320 15:27:44.771234 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_6a46b3258ceb4784a9b3ed584b7fcae1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:44.772006 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a46b3258ceb4784a9b3ed584b7fcae1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:44.770231 28734 raft_consensus.cc:359] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.772225 28734 raft_consensus.cc:385] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.772303 28734 raft_consensus.cc:740] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.772869 28734 consensus_queue.cc:260] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.773226 28890 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_7397a25b5cf94613bd1970d6483c2cd0 [id=a6da90b5243e4ddc93342b80dac5e784] alter complete (version 2)
I20260320 15:27:44.773523 28734 ts_tablet_manager.cc:1434] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:44.773794 28734 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.776090 30257 raft_consensus.cc:493] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.776239 30257 raft_consensus.cc:515] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.777206 30257 leader_election.cc:290] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:44.777962 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.778707 28904 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 120) took 2978 ms. Trace:
I20260320 15:27:44.778833 28904 rpcz_store.cc:276] 0320 15:27:41.799933 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.733775 (+2933842us) service_pool.cc:225] Handling call
0320 15:27:44.734138 (+ 363us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:44.734347 (+ 209us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:44.734555 (+ 208us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:44.734629 (+ 74us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:44.755399 (+ 20770us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:44.778654 (+ 23255us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.735634 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.735747 (+ 113us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.735755 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:44.735757 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.735858 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.735874 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.735878 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.735944 (+ 66us) tablet.cc:700] Row locks acquired
0320 15:27:44.735947 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.735979 (+ 32us) write_op.cc:270] Start()
0320 15:27:44.736012 (+ 33us) write_op.cc:276] Timestamp: P: 1774020464735926 usec, L: 0
0320 15:27:44.736016 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.736195 (+ 179us) log.cc:844] Serialized 405 byte log entry
0320 15:27:44.746045 (+ 9850us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.746182 (+ 137us) write_op.cc:301] APPLY: starting
0320 15:27:44.746201 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.746245 (+ 44us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.746308 (+ 63us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.746363 (+ 55us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.746403 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.746411 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.746495 (+ 84us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.746567 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.746585 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:44.746592 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.746661 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.746664 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":475,"apply.run_wall_time_us":473,"num_ops":1,"prepare.queue_time_us":454,"prepare.run_cpu_time_us":684,"prepare.run_wall_time_us":684,"raft.queue_time_us":66,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":220,"replication_time_us":9993}]]}
I20260320 15:27:44.779768 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_59eba3ec25134b28aeb775da692dae4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:44.780344 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59eba3ec25134b28aeb775da692dae4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:44.784544 28890 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd] alter complete (version 2)
W20260320 15:27:44.788609 28906 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d94b4a6d75804c3e947af4f588ef8f5a consensus state has no leader
I20260320 15:27:44.788841 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 118) took 3102 ms. Trace:
I20260320 15:27:44.788928 28906 rpcz_store.cc:276] 0320 15:27:41.686226 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.296009 (+2609783us) service_pool.cc:225] Handling call
0320 15:27:44.296554 (+ 545us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:44.296839 (+ 285us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:44.297083 (+ 244us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:44.297160 (+ 77us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:44.307636 (+ 10476us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:44.788774 (+481138us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.297866 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.298005 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.298013 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:44.298018 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:44.298139 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.298157 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.298164 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.298253 (+ 89us) tablet.cc:700] Row locks acquired
0320 15:27:44.298257 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.298300 (+ 43us) write_op.cc:270] Start()
0320 15:27:44.298339 (+ 39us) write_op.cc:276] Timestamp: P: 1774020464298282 usec, L: 0
0320 15:27:44.298344 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.298569 (+ 225us) log.cc:844] Serialized 405 byte log entry
0320 15:27:44.302581 (+ 4012us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.302805 (+ 224us) write_op.cc:301] APPLY: starting
0320 15:27:44.302823 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.302868 (+ 45us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.302900 (+ 32us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.302953 (+ 53us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.302985 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.302993 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.303064 (+ 71us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.303110 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.303126 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:44.303131 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.303201 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.303205 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":170,"apply.run_cpu_time_us":390,"apply.run_wall_time_us":390,"num_ops":1,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":826,"prepare.run_wall_time_us":825,"raft.queue_time_us":327,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"replication_time_us":4202}]]}
I20260320 15:27:44.789831 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_76bddc5536ff43e895889c5c30c5068f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:44.790441 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76bddc5536ff43e895889c5c30c5068f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:44.791375 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:44.800165 30454 raft_consensus.cc:493] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.800339 30454 raft_consensus.cc:515] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.801069 30454 leader_election.cc:290] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.802923 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 123) took 2438 ms. Trace:
I20260320 15:27:44.803042 28890 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_b23b6416f43846bab7b604e6b7935bad [id=4654af060c0c4bc19f8d93dba5db8dd0] alter complete (version 1)
I20260320 15:27:44.803062 28906 rpcz_store.cc:276] 0320 15:27:42.364391 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.789584 (+2425193us) service_pool.cc:225] Handling call
0320 15:27:44.790540 (+ 956us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:44.791016 (+ 476us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:44.802605 (+ 11589us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:44.802718 (+ 113us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:44.802829 (+ 111us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.792091 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.792194 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.792200 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:44.792202 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.792301 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.792312 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.792317 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.792388 (+ 71us) tablet.cc:700] Row locks acquired
0320 15:27:44.792390 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.792425 (+ 35us) write_op.cc:270] Start()
0320 15:27:44.792484 (+ 59us) write_op.cc:276] Timestamp: P: 1774020464792406 usec, L: 0
0320 15:27:44.792488 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.792662 (+ 174us) log.cc:844] Serialized 433 byte log entry
0320 15:27:44.800596 (+ 7934us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.801405 (+ 809us) write_op.cc:301] APPLY: starting
0320 15:27:44.801421 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.801462 (+ 41us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.801491 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.801631 (+ 140us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.801662 (+ 31us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:44.801669 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:44.801739 (+ 70us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.801784 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.801810 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:44.801816 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":771,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":479,"num_ops":2,"prepare.queue_time_us":556,"prepare.run_cpu_time_us":646,"prepare.run_wall_time_us":646,"replication_time_us":8088,"threads_started":1}]]}
I20260320 15:27:44.734972 30155 raft_consensus.cc:493] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.737828 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 147) took 10544 ms. Trace:
I20260320 15:27:44.731770 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet e2243e6811744b7cba99a487f8e8bb83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.752907 30394 raft_consensus.cc:493] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.810578 30394 raft_consensus.cc:515] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:44.783170 28524 consensus_peers.cc:597] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ccbf4f32636463f8e1e005a629b92cc. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.810397 29640 rpcz_store.cc:276] 0320 15:27:34.202467 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.202526 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:44.696730 (+10494204us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:44.696741 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.712477 (+ 15736us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.724721 (+ 12244us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.737814 (+ 13093us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10484915}
I20260320 15:27:44.811049 29640 tablet_replica.cc:333] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.811290 29640 raft_consensus.cc:2243] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.811376 30394 leader_election.cc:290] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.781759 29939 raft_consensus.cc:493] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.811472 29640 raft_consensus.cc:2272] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.811581 29939 raft_consensus.cc:515] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.812029 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.812111 29640 ts_tablet_manager.cc:1916] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.812144 29939 leader_election.cc:290] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.812232 28441 raft_consensus.cc:2468] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.793821 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.802088 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:44.812876 28382 leader_election.cc:343] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:44.812872 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7feb8477ebf841a19977a434f365f537 (table default.Table_fdd90d848d4846dcb37f6b50bce8156b [id=4d6287f82e004f15b09b7afa88fb47d5]) successfully deleted
I20260320 15:27:44.813129 28524 leader_election.cc:304] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.813588 29939 raft_consensus.cc:2804] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.813659 29939 raft_consensus.cc:493] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.813719 29939 raft_consensus.cc:3060] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.813772 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.813954 28441 raft_consensus.cc:2468] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.814718 28524 leader_election.cc:304] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.815022 30394 raft_consensus.cc:2804] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.815114 30394 raft_consensus.cc:493] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.815174 30394 raft_consensus.cc:3060] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.817435 29939 raft_consensus.cc:515] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.817988 30394 raft_consensus.cc:515] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.818262 29939 leader_election.cc:290] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.818516 30394 leader_election.cc:290] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.819762 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.819968 28444 raft_consensus.cc:3060] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.810431 30155 raft_consensus.cc:515] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.820595 30155 leader_election.cc:290] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.822753 28444 raft_consensus.cc:2468] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.823263 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.823432 28441 raft_consensus.cc:3060] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.826143 28441 raft_consensus.cc:2468] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.826186 28904 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 122) took 2492 ms. Trace:
I20260320 15:27:44.826339 28904 rpcz_store.cc:276] 0320 15:27:42.333976 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.779511 (+2445535us) service_pool.cc:225] Handling call
0320 15:27:44.783317 (+ 3806us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:44.783899 (+ 582us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:44.825785 (+ 41886us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:44.825923 (+ 138us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:44.826049 (+ 126us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.785933 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.786066 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.786072 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:44.786075 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.786170 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.786196 (+ 26us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.786200 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.786282 (+ 82us) tablet.cc:700] Row locks acquired
0320 15:27:44.786285 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.786322 (+ 37us) write_op.cc:270] Start()
0320 15:27:44.786358 (+ 36us) write_op.cc:276] Timestamp: P: 1774020464786302 usec, L: 0
0320 15:27:44.786362 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.786568 (+ 206us) log.cc:844] Serialized 433 byte log entry
0320 15:27:44.794898 (+ 8330us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.795017 (+ 119us) write_op.cc:301] APPLY: starting
0320 15:27:44.795035 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.795110 (+ 75us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.795158 (+ 48us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.795267 (+ 109us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.795312 (+ 45us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:44.795319 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:44.795403 (+ 84us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.795482 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.795504 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:44.795511 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.795618 (+ 107us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.795622 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":589,"apply.run_wall_time_us":588,"num_ops":2,"prepare.queue_time_us":1518,"prepare.run_cpu_time_us":733,"prepare.run_wall_time_us":734,"raft.queue_time_us":151,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":291,"replication_time_us":8503}]]}
I20260320 15:27:44.826938 28524 leader_election.cc:304] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.827282 30155 raft_consensus.cc:2804] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.827390 30155 raft_consensus.cc:697] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.827661 30155 consensus_queue.cc:237] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.829794 29640 ts_tablet_manager.cc:1929] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.829921 29640 log.cc:1199] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:44.830181 28524 leader_election.cc:304] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.831820 30355 raft_consensus.cc:2804] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.831944 30355 raft_consensus.cc:697] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.832156 30355 consensus_queue.cc:237] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.832625 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.833030 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.833206 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.834057 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd with delete_type TABLET_DATA_DELETED (replaced by tablet d78ab2f7e0d7413ea9049dba9cbd8633) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.834367 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet b03e7555b9a646b599acb0c7dff90c9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:22 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.834920 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.835304 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.835421 28736 tablet_bootstrap.cc:654] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.835507 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet ffe278739bbc4ddab848af9c25241380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.836118 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.836453 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bf5e17dea5445d9909a229fa8315348 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.836889 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet c3bc4dd47817490d88725127b456cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.839290 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.839587 28441 raft_consensus.cc:2468] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.840121 28524 leader_election.cc:304] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.840543 29939 raft_consensus.cc:2804] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.840723 29939 raft_consensus.cc:493] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.840852 29939 raft_consensus.cc:3060] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.842329 28734 tablet_bootstrap.cc:654] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.844336 29939 raft_consensus.cc:515] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.846212 30395 raft_consensus.cc:493] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.846890 30395 raft_consensus.cc:515] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.847258 29939 leader_election.cc:290] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.848899 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.849107 28441 raft_consensus.cc:3060] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.852752 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.853024 28444 raft_consensus.cc:2393] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:44.853235 30466 raft_consensus.cc:493] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.853408 30466 raft_consensus.cc:515] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.853928 30395 leader_election.cc:290] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.854060 30466 leader_election.cc:290] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:44.854703 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.854987 28444 raft_consensus.cc:2393] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:44.856307 28441 raft_consensus.cc:2468] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.856783 28524 leader_election.cc:304] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.857991 30395 raft_consensus.cc:2804] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.858290 30395 raft_consensus.cc:697] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.858642 30395 consensus_queue.cc:237] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.859678 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet ea6ff59d2ffe4d579e6db2375eb11414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.860647 28736 tablet_bootstrap.cc:492] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.861019 30454 consensus_queue.cc:1048] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.861230 28736 ts_tablet_manager.cc:1403] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.000s
I20260320 15:27:44.862406 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fa5f6fe801b4452bf294a9ae23bb163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.864491 29643 ts_tablet_manager.cc:1929] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.864642 29643 log.cc:1199] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8d92e17cea044541968356ae993be1e7
I20260320 15:27:44.865149 29643 ts_tablet_manager.cc:1950] T 8d92e17cea044541968356ae993be1e7 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:44.866200 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 169) took 9276 ms. Trace:
I20260320 15:27:44.810503 28733 raft_consensus.cc:385] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.866344 29643 rpcz_store.cc:276] 0320 15:27:35.590017 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.590133 (+ 116us) service_pool.cc:225] Handling call
0320 15:27:44.682128 (+9091995us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e17cea044541968356ae993be1e7
0320 15:27:44.703195 (+ 21067us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x61800000c918. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff37d3529ad 00007ff37d353a1e 00007ff390fe279d 00007ff390f614c3 00007ff38d17ce26 00007ff38d17e1b9 00007ff38d1b1b44 00007ff38d192560 00007ff38d1922ed 0000000000a8b58c
0320 15:27:44.719118 (+ 15923us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.811265 (+ 92147us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.863328 (+ 52063us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.866171 (+ 2843us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":48132352,"tablet-delete.queue_time_us":9091707}
I20260320 15:27:44.866585 28733 raft_consensus.cc:740] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.867060 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]) successfully deleted
I20260320 15:27:44.867033 28733 consensus_queue.cc:260] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.867784 28733 ts_tablet_manager.cc:1434] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.130s user 0.000s sys 0.002s
I20260320 15:27:44.868083 28733 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.869573 29678 ts_tablet_manager.cc:1929] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:44.869894 29678 log.cc:1199] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/54266dd376164673bffe829dd515b83d
I20260320 15:27:44.869930 28733 tablet_bootstrap.cc:654] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.870385 29678 ts_tablet_manager.cc:1950] T 54266dd376164673bffe829dd515b83d P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:44.874114 29643 tablet_replica.cc:333] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:44.879109 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a with delete_type TABLET_DATA_DELETED (replaced by tablet fca60a0f372f4d1194d754d124b78ff8) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.879392 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:44.883239 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae3422cf33324ab89a2b44e27cae0eb2 already in progress: creating tablet
W20260320 15:27:44.883509 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because the tablet was not found. No further retry: Not found: Tablet not found: 2165371f4b5448c7a2625926082ee4c6
W20260320 15:27:44.883800 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b34048a722c3485490637965c60cc5dc because the tablet was not found. No further retry: Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
I20260320 15:27:44.885712 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 195) took 7802 ms. Trace:
I20260320 15:27:44.885844 29678 rpcz_store.cc:276] 0320 15:27:37.083058 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.083101 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:44.697314 (+7614213us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:44.697322 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.824206 (+126884us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.869449 (+ 45243us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.885674 (+ 16225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7614038}
I20260320 15:27:44.886332 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 196) took 7802 ms. Trace:
I20260320 15:27:44.886436 29678 rpcz_store.cc:276] 0320 15:27:37.083426 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.083469 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:44.886176 (+7802707us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:44.886192 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.886309 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7802531}
I20260320 15:27:44.886780 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 197) took 7803 ms. Trace:
I20260320 15:27:44.872423 28736 raft_consensus.cc:359] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.882398 28734 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.887639 28734 ts_tablet_manager.cc:1403] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.114s user 0.002s sys 0.002s
I20260320 15:27:44.888880 28734 raft_consensus.cc:359] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.889132 28734 raft_consensus.cc:385] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.889597 28734 raft_consensus.cc:740] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.889976 28734 consensus_queue.cc:260] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.890890 28734 ts_tablet_manager.cc:1434] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:44.891309 28734 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.893501 28734 tablet_bootstrap.cc:654] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.881687 29643 raft_consensus.cc:2243] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:44.896088 29643 raft_consensus.cc:2272] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.896739 29643 ts_tablet_manager.cc:1916] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.898701 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7e3a6fa7dc74de286177a285dd3d877" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:44.898936 28575 raft_consensus.cc:2393] T c7e3a6fa7dc74de286177a285dd3d877 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:44.887349 28736 raft_consensus.cc:385] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.901393 30466 raft_consensus.cc:493] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.903877 28736 raft_consensus.cc:740] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.903924 30466 raft_consensus.cc:515] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.904074 28736 consensus_queue.cc:260] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.904522 30466 leader_election.cc:290] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:44.904685 28736 ts_tablet_manager.cc:1434] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
I20260320 15:27:44.904961 28736 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:44.902758 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.903739 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a3d5e4ff8f42409482af49c636be0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.903663 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet e2243e6811744b7cba99a487f8e8bb83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:44.905675 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4380f86ceb48b8b93eb2a5b9b302c7" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:44.897507 28351 heartbeater.cc:344] Connected to a master server at 127.27.74.61:44739
W20260320 15:27:44.906998 28240 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:44.907276 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 7feb8477ebf841a19977a434f365f537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.908073 28736 tablet_bootstrap.cc:654] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.886853 29678 rpcz_store.cc:276] 0320 15:27:37.083778 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.083820 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:44.886707 (+7802887us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:44.886717 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.886768 (+ 51us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7802753}
I20260320 15:27:44.911674 29678 tablet_replica.cc:333] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:44.911900 29678 raft_consensus.cc:2243] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.912271 29678 raft_consensus.cc:2272] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.912999 28736 tablet_bootstrap.cc:492] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:44.913123 28736 ts_tablet_manager.cc:1403] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260320 15:27:44.913955 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e13c75a376c47c9a0083173865ce4fa" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:44.914124 28297 raft_consensus.cc:3060] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.915239 28733 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.915405 28733 ts_tablet_manager.cc:1403] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20260320 15:27:44.916396 28733 raft_consensus.cc:359] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.916617 28733 raft_consensus.cc:385] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.916682 28733 raft_consensus.cc:740] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.916949 28733 consensus_queue.cc:260] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.918632 30344 consensus_queue.cc:1048] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:44.919440 28297 raft_consensus.cc:2468] T 7e13c75a376c47c9a0083173865ce4fa P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:44.925489 28524 consensus_peers.cc:597] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0a31d385b81456fb46ea52083874fcf. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.932559 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 9176e734212849aab8d99473de02cc79 with delete_type TABLET_DATA_DELETED (replaced by tablet dd315b5d49104d679b03515b65ee30ac) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.933454 28733 ts_tablet_manager.cc:1434] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
I20260320 15:27:44.933506 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 7be03035dc3d44f9aa5b43b49281a940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:44.933787 28733 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:44.939499 29640 ts_tablet_manager.cc:1950] T 43a0fadc7dac4e21ad034e4cd2d70826 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:44.914052 28736 raft_consensus.cc:359] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.941483 28736 raft_consensus.cc:385] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.941586 28736 raft_consensus.cc:740] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.941808 28736 consensus_queue.cc:260] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:44.942422 28736 ts_tablet_manager.cc:1434] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.029s user 0.002s sys 0.000s
I20260320 15:27:44.942736 28736 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:44.943202 28240 consensus_peers.cc:597] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:44.943681 29939 raft_consensus.cc:493] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:44.943884 29939 raft_consensus.cc:515] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.945209 29939 leader_election.cc:290] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.945498 28733 tablet_bootstrap.cc:654] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.945652 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:44.945844 28444 raft_consensus.cc:2468] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:44.946240 28524 leader_election.cc:304] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.949033 29939 raft_consensus.cc:2804] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:44.949101 28734 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:44.949225 29939 raft_consensus.cc:493] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:44.949229 28734 ts_tablet_manager.cc:1403] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20260320 15:27:44.949366 29939 raft_consensus.cc:3060] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.950079 28734 raft_consensus.cc:359] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.950266 28734 raft_consensus.cc:385] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.950321 28734 raft_consensus.cc:740] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.950536 28734 consensus_queue.cc:260] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.951005 28736 tablet_bootstrap.cc:654] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.951068 28734 ts_tablet_manager.cc:1434] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:44.951350 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.953639 29939 raft_consensus.cc:515] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.955477 28734 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:44.957731 28734 tablet_bootstrap.cc:654] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:44.958382 28890 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 107) took 2384 ms (client timeout 1000 ms). Trace:
I20260320 15:27:44.959292 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:44.959515 28444 raft_consensus.cc:3060] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.967067 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:44.967410 28444 raft_consensus.cc:2468] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:44.967563 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 148) took 10756 ms. Trace:
I20260320 15:27:44.967682 29640 rpcz_store.cc:276] 0320 15:27:34.217519 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.217575 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:44.810986 (+10593411us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:44.811000 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.815824 (+ 4824us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.829674 (+ 13850us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:44.967153 (+137479us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11520,"tablet-delete.queue_time_us":10593008}
I20260320 15:27:44.968331 28524 leader_election.cc:304] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:44.968639 30395 raft_consensus.cc:2804] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:44.968772 30395 raft_consensus.cc:697] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:44.969009 30395 consensus_queue.cc:237] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.970242 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 43a0fadc7dac4e21ad034e4cd2d70826 (table default.Table_051864d5d9984140b3ce79ec28717420 [id=e4ae921b3eef49048b192bf1393aff45]) successfully deleted
I20260320 15:27:44.975570 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_786940b8acbe4d13944ef32c8b08cbba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:44.976573 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_786940b8acbe4d13944ef32c8b08cbba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:44.978420 29640 tablet_replica.cc:333] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:44.981627 29678 ts_tablet_manager.cc:1916] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:44.984221 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.984838 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:44.984913 29640 raft_consensus.cc:2243] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:44.985271 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea3bf14653f4555acfed1577a79ee4f" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:44.983659 28890 rpcz_store.cc:269] 0320 15:27:42.573454 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.116893 (+543439us) service_pool.cc:225] Handling call
0320 15:27:44.000305 (+883412us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.000319 (+ 14us) write_op.cc:454] Released schema lock
0320 15:27:44.958268 (+957949us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.308697 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.308837 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.308846 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:43.308858 (+ 12us) tablet.cc:661] Decoding operations
0320 15:27:43.308982 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.308998 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.309003 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:43.309067 (+ 64us) tablet.cc:700] Row locks acquired
0320 15:27:43.309070 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:43.309110 (+ 40us) write_op.cc:270] Start()
0320 15:27:43.309149 (+ 39us) write_op.cc:276] Timestamp: P: 1774020463309085 usec, L: 0
0320 15:27:43.309153 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.309376 (+ 223us) log.cc:844] Serialized 527 byte log entry
0320 15:27:43.312947 (+ 3571us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.313072 (+ 125us) write_op.cc:301] APPLY: starting
0320 15:27:43.313094 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.313143 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.313183 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.313254 (+ 71us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.313290 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.313297 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:43.313408 (+ 111us) log.cc:844] Serialized 23 byte log entry
0320 15:27:43.313474 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.313498 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:43.313505 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.313588 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.313592 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:43.984624 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.984721 (+ 97us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.984728 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:43.984731 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:43.984842 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.984859 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.984863 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:43.984933 (+ 70us) tablet.cc:700] Row locks acquired
0320 15:27:43.984935 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:43.984971 (+ 36us) write_op.cc:270] Start()
0320 15:27:43.985008 (+ 37us) write_op.cc:276] Timestamp: P: 1774020463984936 usec, L: 0
0320 15:27:43.985012 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.985217 (+ 205us) log.cc:844] Serialized 356 byte log entry
0320 15:27:43.989546 (+ 4329us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.989673 (+ 127us) write_op.cc:301] APPLY: starting
0320 15:27:43.989695 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.989766 (+ 71us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.989876 (+ 110us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.989953 (+ 77us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.989996 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.990004 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:43.990101 (+ 97us) log.cc:844] Serialized 23 byte log entry
0320 15:27:43.990177 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.990208 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:43.990216 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.743429 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.743591 (+ 162us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.743598 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:44.743601 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.743705 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.743718 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.743723 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.743789 (+ 66us) tablet.cc:700] Row locks acquired
0320 15:27:44.743791 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.743835 (+ 44us) write_op.cc:270] Start()
0320 15:27:44.743872 (+ 37us) write_op.cc:276] Timestamp: P: 1774020464743814 usec, L: 0
0320 15:27:44.743876 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.744085 (+ 209us) log.cc:844] Serialized 356 byte log entry
0320 15:27:44.752121 (+ 8036us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.752254 (+ 133us) write_op.cc:301] APPLY: starting
0320 15:27:44.752273 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.752315 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.752350 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.752405 (+ 55us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.752437 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.752445 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.752537 (+ 92us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.752598 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.752618 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:44.752625 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.752714 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.752718 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:44.774447 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.774566 (+ 119us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.774573 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:44.774576 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.774691 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.774706 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.774711 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.774786 (+ 75us) tablet.cc:700] Row locks acquired
0320 15:27:44.774788 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.774823 (+ 35us) write_op.cc:270] Start()
0320 15:27:44.774857 (+ 34us) write_op.cc:276] Timestamp: P: 1774020464774805 usec, L: 0
0320 15:27:44.774860 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.775075 (+ 215us) log.cc:844] Serialized 356 byte log entry
0320 15:27:44.781260 (+ 6185us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.781386 (+ 126us) write_op.cc:301] APPLY: starting
0320 15:27:44.781405 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.781450 (+ 45us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.781487 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.781546 (+ 59us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.781582 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.781590 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:44.781675 (+ 85us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.781741 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.781764 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:44.781770 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.781839 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.781843 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:44.787175 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.787287 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.787292 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.787295 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.787386 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.787397 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.787401 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.787468 (+ 67us) tablet.cc:700] Row locks acquired
0320 15:27:44.787471 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.787503 (+ 32us) write_op.cc:270] Start()
0320 15:27:44.787559 (+ 56us) write_op.cc:276] Timestamp: P: 1774020464787484 usec, L: 0
0320 15:27:44.787562 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.787902 (+ 340us) log.cc:844] Serialized 356 byte log entry
0320 15:27:44.800518 (+ 12616us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.800806 (+ 288us) write_op.cc:301] APPLY: starting
0320 15:27:44.800825 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.800871 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.800908 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.800970 (+ 62us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.801011 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.801020 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:44.801112 (+ 92us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.801197 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.801217 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:44.801224 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.801317 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.801323 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11761,"reported_tablets":179,"child_traces":[["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":502,"apply.run_wall_time_us":501,"num_ops":1,"prepare.queue_time_us":6208,"prepare.run_cpu_time_us":803,"prepare.run_wall_time_us":804,"raft.queue_time_us":336,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":248,"replication_time_us":3757}],["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":602,"apply.run_wall_time_us":602,"num_ops":1,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":692,"prepare.run_wall_time_us":693,"raft.queue_time_us":493,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":287,"replication_time_us":4495}],["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":433,"apply.run_wall_time_us":432,"num_ops":1,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":726,"prepare.run_wall_time_us":728,"replication_time_us":8211}],["op",{"apply.queue_time_us":76,"apply.run_cpu_time_us":444,"apply.run_wall_time_us":443,"num_ops":1,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":736,"prepare.run_wall_time_us":736,"raft.queue_time_us":176,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":281,"replication_time_us":6363}],["op",{"apply.queue_time_us":230,"apply.run_cpu_time_us":499,"apply.run_wall_time_us":499,"mutex_wait_us":1244,"num_ops":1,"prepare.queue_time_us":1288,"prepare.run_cpu_time_us":650,"prepare.run_wall_time_us":796,"replication_time_us":12918}]]}
I20260320 15:27:44.985764 29640 raft_consensus.cc:2272] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:44.957724 29939 leader_election.cc:290] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:44.989298 28733 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:44.989528 28733 ts_tablet_manager.cc:1403] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.002s
I20260320 15:27:44.990269 28733 raft_consensus.cc:359] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.990516 28733 raft_consensus.cc:385] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:44.990607 28733 raft_consensus.cc:740] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:44.990854 28733 consensus_queue.cc:260] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:44.991477 28733 ts_tablet_manager.cc:1434] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:44.993582 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.994194 28296 raft_consensus.cc:3060] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.994904 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8428019c3b074b17aec493c14cc2b911" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:44.995728 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:44.997509 28302 raft_consensus.cc:3060] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:44.998464 28300 raft_consensus.cc:3060] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.004189 28733 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.005848 28733 tablet_bootstrap.cc:654] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:44.991827 29939 consensus_queue.cc:1048] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.008153 29939 raft_consensus.cc:493] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.008438 29939 raft_consensus.cc:515] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.003364 29640 ts_tablet_manager.cc:1916] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.009289 29939 leader_election.cc:290] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.009369 28734 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.009541 28734 ts_tablet_manager.cc:1403] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.055s user 0.012s sys 0.004s
I20260320 15:27:45.009685 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.009891 28446 raft_consensus.cc:2468] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:45.010356 28734 raft_consensus.cc:359] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.010520 28734 raft_consensus.cc:385] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.010566 28734 raft_consensus.cc:740] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.010754 28734 consensus_queue.cc:260] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.011557 28734 ts_tablet_manager.cc:1434] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.011595 28524 leader_election.cc:304] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.011852 28734 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.012130 29939 raft_consensus.cc:2804] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.012262 29939 raft_consensus.cc:493] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.012416 29939 raft_consensus.cc:3060] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.013753 28734 tablet_bootstrap.cc:654] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.013958 28736 tablet_bootstrap.cc:492] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.014122 28736 ts_tablet_manager.cc:1403] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.002s
I20260320 15:27:45.014367 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_a3cd321472b34384aadb6e966f5402da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:45.015636 28733 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.015776 28733 ts_tablet_manager.cc:1403] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20260320 15:27:45.020550 28733 raft_consensus.cc:359] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.020756 28733 raft_consensus.cc:385] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.020813 28733 raft_consensus.cc:740] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:45.021349 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3cd321472b34384aadb6e966f5402da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:45.021462 28733 consensus_queue.cc:260] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.022148 28733 ts_tablet_manager.cc:1434] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260320 15:27:45.022313 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.022456 28733 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.022754 28295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:45.022894 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 54266dd376164673bffe829dd515b83d (table default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]) successfully deleted
I20260320 15:27:45.023056 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:45.024008 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 931f6f88a1d94a40b69f07216b8313c8 because the tablet was not found. No further retry: Not found: Tablet not found: 931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:45.020839 29640 ts_tablet_manager.cc:1929] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.025326 29640 log.cc:1199] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:45.025749 29640 ts_tablet_manager.cc:1950] T 0e9a4e61bab44d768d9adfed00c945e9 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.026176 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 149) took 10808 ms. Trace:
I20260320 15:27:45.026268 29640 rpcz_store.cc:276] 0320 15:27:34.218234 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.218280 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:44.978350 (+10760070us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:44.978364 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.985339 (+ 6975us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x616000133354. stack: 00007f9650f30076 00007f9650f2fe30 00007f964feea3ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f965ce2f7e0 00007f965e180dc3 00007f9660d44a77 00007f9660d77b44 00007f9660d58560 00007f9660d582ed 0000000000a8b58c 00007f9650f75f5f 00007f9650f7927b 00007f9650f790f8
0320 15:27:45.012330 (+ 26991us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.020725 (+ 8395us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.026157 (+ 5432us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":14316288,"tablet-delete.queue_time_us":10759497}
I20260320 15:27:45.027002 29640 tablet_replica.cc:333] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.027220 29640 raft_consensus.cc:2243] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.027606 29640 raft_consensus.cc:2272] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.028198 29640 ts_tablet_manager.cc:1916] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.030157 28733 tablet_bootstrap.cc:654] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.030355 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 0e9a4e61bab44d768d9adfed00c945e9 (table default.Table_c77304a1097d4fb987cc1aa19d0fb284 [id=df66d2858330460bab50d7b946fd30e4]) successfully deleted
I20260320 15:27:45.035298 28736 raft_consensus.cc:359] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.035519 28736 raft_consensus.cc:385] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.035609 28736 raft_consensus.cc:740] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.035789 28736 consensus_queue.cc:260] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.036474 28736 ts_tablet_manager.cc:1434] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.022s user 0.000s sys 0.001s
I20260320 15:27:45.036854 28736 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.039105 28736 tablet_bootstrap.cc:654] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.043605 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet ef09e4afdd1f4fce9caec4dcafa2c50c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.044519 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 533105ad6e054fe3a4db32855bc688bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.047430 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd8f5ccbd35418684fd27459c7a6b9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.047710 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 20a7164db1e0447fb424d8e46b747f52 with delete_type TABLET_DATA_DELETED (replaced by tablet 57cbefe3a19044f78b64db951df5f7c1) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.048003 30466 consensus_queue.cc:1048] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.048576 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 07af28f257d745a1bb4173aad09045d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.049913 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet fc009401bb3d440f9d36e080efef1204 with delete_type TABLET_DATA_DELETED (replaced by tablet f6240d441c624b238f7c13eb3c5fa9fc) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.049971 28734 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.050123 28734 ts_tablet_manager.cc:1403] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.038s user 0.007s sys 0.007s
I20260320 15:27:45.050355 30466 raft_consensus.cc:493] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.050496 30466 raft_consensus.cc:515] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.050912 28734 raft_consensus.cc:359] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.051067 28734 raft_consensus.cc:385] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.051081 29640 ts_tablet_manager.cc:1929] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.051098 30466 leader_election.cc:290] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:45.051119 28734 raft_consensus.cc:740] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.051180 29640 log.cc:1199] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:45.051332 28734 consensus_queue.cc:260] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.051561 29640 ts_tablet_manager.cc:1950] T 3ed29146bff74e86a366d3c6329176f4 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.051892 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 150) took 10831 ms. Trace:
I20260320 15:27:45.051955 28734 ts_tablet_manager.cc:1434] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.052217 28734 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.052166 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.053992 28734 tablet_bootstrap.cc:654] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.051972 29640 rpcz_store.cc:276] 0320 15:27:34.220832 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.220882 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:45.026955 (+10806073us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:45.026964 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.033940 (+ 6976us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.050964 (+ 17024us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.051875 (+ 911us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10805844}
I20260320 15:27:45.056128 29640 tablet_replica.cc:333] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.056241 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 3ed29146bff74e86a366d3c6329176f4 (table default.Table_3618306c0eff4c74935789df59e679f4 [id=721eb95aef9d4576985e266c41624147]) successfully deleted
I20260320 15:27:45.056433 29640 raft_consensus.cc:2243] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.056643 29640 raft_consensus.cc:2272] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.057432 29640 ts_tablet_manager.cc:1916] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.057634 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:45.081054 29640 ts_tablet_manager.cc:1929] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.081235 29640 log.cc:1199] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:45.081658 29640 ts_tablet_manager.cc:1950] T e7bd0616f5604f8fbb8edd8e9c230f39 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.081766 30466 consensus_queue.cc:1048] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.082026 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 153) took 10856 ms. Trace:
I20260320 15:27:45.082135 29640 rpcz_store.cc:276] 0320 15:27:34.225898 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.226538 (+ 640us) service_pool.cc:225] Handling call
0320 15:27:45.056049 (+10829511us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:45.056067 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.065127 (+ 9060us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.080904 (+ 15777us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.082000 (+ 1096us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10826770}
I20260320 15:27:45.082577 29640 tablet_replica.cc:333] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.082726 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet e7bd0616f5604f8fbb8edd8e9c230f39 (table default.Table_17c9ae5b739344f4ab272260023b6a74 [id=30b8ab71f82a42ebbb2976696e1ecb4e]) successfully deleted
I20260320 15:27:45.091321 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 121) took 2928 ms. Trace:
I20260320 15:27:45.091475 28900 rpcz_store.cc:276] 0320 15:27:42.162831 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.770981 (+2608150us) service_pool.cc:225] Handling call
0320 15:27:44.772191 (+ 1210us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:44.772846 (+ 655us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.090988 (+318142us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.091111 (+ 123us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.091227 (+ 116us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.870517 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.870642 (+ 125us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.870649 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:44.870651 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.870759 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.870775 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.870781 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.870866 (+ 85us) tablet.cc:700] Row locks acquired
0320 15:27:44.870868 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.870905 (+ 37us) write_op.cc:270] Start()
0320 15:27:44.870949 (+ 44us) write_op.cc:276] Timestamp: P: 1774020464870879 usec, L: 0
0320 15:27:44.870954 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.969419 (+ 98465us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x6160000a33b8. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612ea4775 00007fe612f8951b 00007fe612f64518 00007fe612f63dc5 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f
0320 15:27:44.969664 (+ 245us) log.cc:844] Serialized 433 byte log entry
0320 15:27:44.969854 (+ 190us) spinlock_profiling.cc:243] Waited 84.3 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:45.090225 (+120371us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.090403 (+ 178us) write_op.cc:301] APPLY: starting
0320 15:27:45.090425 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.090478 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.090480 (+ 2us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.090612 (+ 132us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.090652 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.090659 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.090755 (+ 96us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.090836 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.090862 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.090870 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.090970 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.090974 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":553,"apply.run_wall_time_us":553,"num_ops":2,"prepare.queue_time_us":61544,"prepare.run_cpu_time_us":2677,"prepare.run_wall_time_us":99419,"raft.queue_time_us":1619,"raft.run_cpu_time_us":524,"raft.run_wall_time_us":2865,"replication_time_us":219222,"spinlock_wait_cycles":225797376}]]}
I20260320 15:27:45.082846 29640 raft_consensus.cc:2243] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.092391 29640 raft_consensus.cc:2272] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.092473 30466 raft_consensus.cc:493] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.092649 30466 raft_consensus.cc:515] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.092759 28903 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_5dabb37ba3784a72b87e63c63b6280aa [id=ba68ceb7a1404bcf90d28969990083dc] alter complete (version 2)
I20260320 15:27:45.092985 29640 ts_tablet_manager.cc:1916] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.093539 30466 leader_election.cc:290] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:45.094626 28240 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 8d92e17cea044541968356ae993be1e7
I20260320 15:27:45.096096 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet a185f84c24ea402684aa1cc3628d4f11
I20260320 15:27:45.096676 28905 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 119) took 3388 ms. Trace:
I20260320 15:27:45.096808 28905 rpcz_store.cc:276] 0320 15:27:41.707912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.730172 (+3022260us) service_pool.cc:225] Handling call
0320 15:27:44.730656 (+ 484us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:44.730802 (+ 146us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:44.730865 (+ 63us) catalog_manager.cc:2824] Locking tablets
0320 15:27:44.887079 (+156214us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:45.095738 (+208659us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:45.095858 (+ 120us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:45.095917 (+ 59us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:45.096624 (+ 707us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.970819 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.970939 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.970945 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:44.970948 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.971047 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.971058 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.971062 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.971160 (+ 98us) tablet.cc:700] Row locks acquired
0320 15:27:44.971162 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.971193 (+ 31us) write_op.cc:270] Start()
0320 15:27:44.971221 (+ 28us) write_op.cc:276] Timestamp: P: 1774020464971157 usec, L: 0
0320 15:27:44.971224 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.971381 (+ 157us) log.cc:844] Serialized 725 byte log entry
0320 15:27:45.092367 (+120986us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.094990 (+ 2623us) write_op.cc:301] APPLY: starting
0320 15:27:45.095017 (+ 27us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.095091 (+ 74us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.095129 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.095210 (+ 81us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.095251 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.095258 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.095390 (+ 132us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.095481 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.095512 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:45.095521 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":83,"child_traces":[["op",{"apply.queue_time_us":2524,"apply.run_cpu_time_us":668,"apply.run_wall_time_us":1010,"num_ops":2,"prepare.queue_time_us":82613,"prepare.run_cpu_time_us":5765,"prepare.run_wall_time_us":93084,"raft.queue_time_us":3028,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":121107,"thread_start_us":92415,"threads_started":1}]]}
I20260320 15:27:45.098905 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet a185f84c24ea402684aa1cc3628d4f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:45.099208 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet a185f84c24ea402684aa1cc3628d4f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.099190 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_5228be23ce9740aa912d56fce56ec7ed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:45.100021 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5228be23ce9740aa912d56fce56ec7ed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:45.102237 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a4e61bab44d768d9adfed00c945e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.103021 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet e7bd0616f5604f8fbb8edd8e9c230f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.106698 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 43a0fadc7dac4e21ad034e4cd2d70826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.106948 28555 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed29146bff74e86a366d3c6329176f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:24 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:45.110519 28903 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 123) took 1323 ms (client timeout 1000 ms). Trace:
W20260320 15:27:45.112895 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 101) took 1059 ms (client timeout 1000 ms). Trace:
W20260320 15:27:45.113113 28889 rpcz_store.cc:269] 0320 15:27:44.053309 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.088745 (+ 35436us) service_pool.cc:225] Handling call
0320 15:27:45.112850 (+1024105us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.633329 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.633467 (+ 138us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.633477 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:44.633482 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:44.633797 (+ 315us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.633815 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.633820 (+ 5us) tablet.cc:684] Acquiring locks for 7 operations
0320 15:27:44.633982 (+ 162us) tablet.cc:700] Row locks acquired
0320 15:27:44.633986 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.634036 (+ 50us) write_op.cc:270] Start()
0320 15:27:44.634075 (+ 39us) write_op.cc:276] Timestamp: P: 1774020464633987 usec, L: 0
0320 15:27:44.634080 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.634349 (+ 269us) log.cc:844] Serialized 2772 byte log entry
0320 15:27:44.639949 (+ 5600us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.640108 (+ 159us) write_op.cc:301] APPLY: starting
0320 15:27:44.640129 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.640250 (+ 121us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.640279 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.640440 (+ 161us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.640489 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0320 15:27:44.640507 (+ 18us) write_op.cc:312] APPLY: finished
0320 15:27:44.640633 (+ 126us) log.cc:844] Serialized 59 byte log entry
0320 15:27:44.640708 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.640741 (+ 33us) write_op.cc:454] Released schema lock
0320 15:27:44.640752 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.640845 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.640849 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":192,"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":732,"apply.run_wall_time_us":731,"num_ops":7,"prepare.queue_time_us":73,"prepare.run_cpu_time_us":1152,"prepare.run_wall_time_us":1152,"raft.queue_time_us":384,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":335,"replication_time_us":5399}]]}
I20260320 15:27:45.113991 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_488a72bf7d2e4ab0a4c050fdb1e707c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:45.110697 28903 rpcz_store.cc:269] 0320 15:27:43.786793 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.797771 (+ 10978us) service_pool.cc:225] Handling call
0320 15:27:43.958687 (+160916us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.110443 (+1151756us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:44.288886 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.289078 (+ 192us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.289088 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:44.289093 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:44.289225 (+ 132us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.289243 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.289262 (+ 19us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.289347 (+ 85us) tablet.cc:700] Row locks acquired
0320 15:27:44.289351 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.289394 (+ 43us) write_op.cc:270] Start()
0320 15:27:44.289443 (+ 49us) write_op.cc:276] Timestamp: P: 1774020464289377 usec, L: 0
0320 15:27:44.289449 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.289733 (+ 284us) log.cc:844] Serialized 527 byte log entry
0320 15:27:44.294169 (+ 4436us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.294317 (+ 148us) write_op.cc:301] APPLY: starting
0320 15:27:44.294338 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.294386 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.294419 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.294487 (+ 68us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.294523 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.294534 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:44.294632 (+ 98us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.294702 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.294725 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:44.294731 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:44.294819 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.294826 (+ 7us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:44.970002 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.970159 (+ 157us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.970167 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:44.970170 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.970310 (+ 140us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.970324 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.970329 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.970405 (+ 76us) tablet.cc:700] Row locks acquired
0320 15:27:44.970407 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.970447 (+ 40us) write_op.cc:270] Start()
0320 15:27:44.970488 (+ 41us) write_op.cc:276] Timestamp: P: 1774020464970410 usec, L: 0
0320 15:27:44.970492 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:44.970690 (+ 198us) log.cc:844] Serialized 356 byte log entry
0320 15:27:45.090324 (+119634us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.091054 (+ 730us) write_op.cc:301] APPLY: starting
0320 15:27:45.091067 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.091102 (+ 35us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.091146 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.091195 (+ 49us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.091219 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.091225 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:45.091301 (+ 76us) log.cc:844] Serialized 23 byte log entry
0320 15:27:45.091364 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.091381 (+ 17us) write_op.cc:454] Released schema lock
0320 15:27:45.091387 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":32473,"reported_tablets":185,"spinlock_wait_cycles":332450560,"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":487,"apply.run_wall_time_us":486,"num_ops":1,"prepare.queue_time_us":70,"prepare.run_cpu_time_us":1103,"prepare.run_wall_time_us":1102,"raft.queue_time_us":808,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":321,"replication_time_us":4682}],["op",{"apply.queue_time_us":693,"apply.run_cpu_time_us":395,"apply.run_wall_time_us":395,"num_ops":1,"prepare.queue_time_us":96076,"prepare.run_cpu_time_us":810,"prepare.run_wall_time_us":810,"raft.queue_time_us":3677,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":4,"replication_time_us":119811}]]}
I20260320 15:27:45.114516 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_467ca42537f0421bacf771620a950e06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:45.115248 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_467ca42537f0421bacf771620a950e06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:45.116034 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_164db1bde0574cd5ad200b6c2c4bb557"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:45.116585 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_164db1bde0574cd5ad200b6c2c4bb557 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260320 15:27:45.117235 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_488a72bf7d2e4ab0a4c050fdb1e707c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:45.121047 30466 consensus_queue.cc:1048] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.123327 30456 raft_consensus.cc:493] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.123620 30456 raft_consensus.cc:515] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.125000 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.125742 28382 leader_election.cc:343] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.129272 29640 ts_tablet_manager.cc:1929] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.129412 29640 log.cc:1199] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/442ca186c9d4449dade39271dec5c608
I20260320 15:27:45.129814 29640 ts_tablet_manager.cc:1950] T 442ca186c9d4449dade39271dec5c608 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.130158 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 154) took 10904 ms. Trace:
I20260320 15:27:45.130263 29640 rpcz_store.cc:276] 0320 15:27:34.226199 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.226565 (+ 366us) service_pool.cc:225] Handling call
0320 15:27:45.082520 (+10855955us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:45.082531 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.125946 (+ 43415us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.129137 (+ 3191us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.130139 (+ 1002us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10853093}
I20260320 15:27:45.130796 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 442ca186c9d4449dade39271dec5c608 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
I20260320 15:27:45.130847 30572 tablet.cc:1723] T 23eda54577404cc6900cb3ae348604d4 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:45.137724 30571 tablet.cc:1723] T 23eda54577404cc6900cb3ae348604d4 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260320 15:27:45.110471 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 99) took 2502 ms (client timeout 1000 ms). Trace:
W20260320 15:27:45.138890 28902 rpcz_store.cc:269] 0320 15:27:42.608078 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.118979 (+510901us) service_pool.cc:225] Handling call
0320 15:27:45.110410 (+1991431us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:43.728885 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.729068 (+ 183us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.729078 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:43.729082 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:43.729298 (+ 216us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.729317 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.729322 (+ 5us) tablet.cc:684] Acquiring locks for 3 operations
0320 15:27:43.729480 (+ 158us) tablet.cc:700] Row locks acquired
0320 15:27:43.729484 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:43.729618 (+ 134us) write_op.cc:270] Start()
0320 15:27:43.729671 (+ 53us) write_op.cc:276] Timestamp: P: 1774020463729598 usec, L: 0
0320 15:27:43.729677 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.729896 (+ 219us) log.cc:844] Serialized 1275 byte log entry
0320 15:27:43.743749 (+ 13853us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.743926 (+ 177us) write_op.cc:301] APPLY: starting
0320 15:27:43.743948 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.744026 (+ 78us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.744064 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.744160 (+ 96us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.744213 (+ 53us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0320 15:27:43.744220 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:43.744319 (+ 99us) log.cc:844] Serialized 35 byte log entry
0320 15:27:43.744392 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.744421 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:43.744429 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.744509 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.744513 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0320 15:27:43.912982 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:43.913148 (+ 166us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:43.913157 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:43.913162 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:43.913331 (+ 169us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:43.913350 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:43.913357 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:43.913444 (+ 87us) tablet.cc:700] Row locks acquired
0320 15:27:43.913448 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:43.913501 (+ 53us) write_op.cc:270] Start()
0320 15:27:43.913549 (+ 48us) write_op.cc:276] Timestamp: P: 1774020463913466 usec, L: 0
0320 15:27:43.913553 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:43.963033 (+ 49480us) log.cc:844] Serialized 356 byte log entry
0320 15:27:43.963253 (+ 220us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:43.968867 (+ 5614us) op_driver.cc:464] REPLICATION: finished
0320 15:27:43.969048 (+ 181us) write_op.cc:301] APPLY: starting
0320 15:27:43.969070 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:43.969120 (+ 50us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:43.969122 (+ 2us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:43.969178 (+ 56us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:43.969214 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:43.969222 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:43.969312 (+ 90us) log.cc:844] Serialized 23 byte log entry
0320 15:27:43.969376 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.969395 (+ 19us) write_op.cc:454] Released schema lock
0320 15:27:43.969403 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:43.969481 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:43.969484 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22536,"reported_tablets":194,"child_traces":[["op",{"apply.queue_time_us":115,"apply.run_cpu_time_us":575,"apply.run_wall_time_us":574,"num_ops":3,"prepare.queue_time_us":467,"prepare.run_cpu_time_us":1149,"prepare.run_wall_time_us":1151,"raft.queue_time_us":408,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":248,"replication_time_us":14030}],["op",{"apply.queue_time_us":128,"apply.run_cpu_time_us":417,"apply.run_wall_time_us":415,"num_ops":1,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":1543,"prepare.run_wall_time_us":50362,"raft.queue_time_us":1091,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"replication_time_us":55276,"spinlock_wait_cycles":113307904}]]}
I20260320 15:27:45.143321 30155 raft_consensus.cc:493] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.143491 30155 raft_consensus.cc:515] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.143777 28902 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_a2fb8a623b1243d6910bf70670373731"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:45.144081 30155 leader_election.cc:290] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:45.144464 28902 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2fb8a623b1243d6910bf70670373731 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:45.146410 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.146659 28439 raft_consensus.cc:2468] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:45.147296 28524 leader_election.cc:304] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.147894 30155 raft_consensus.cc:2804] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.147981 30155 raft_consensus.cc:493] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.148056 30155 raft_consensus.cc:3060] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.148155 29640 tablet_replica.cc:333] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.148415 29640 raft_consensus.cc:2243] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.148797 29640 raft_consensus.cc:2272] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.149488 29640 ts_tablet_manager.cc:1916] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.150029 29643 ts_tablet_manager.cc:1929] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.150132 29643 log.cc:1199] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:45.150509 29643 ts_tablet_manager.cc:1950] T 27ccc693e7704650bd77e0c4ca11862e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:45.150712 30155 raft_consensus.cc:515] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.150825 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 173) took 9515 ms. Trace:
I20260320 15:27:45.150895 29643 rpcz_store.cc:276] 0320 15:27:35.635347 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.635494 (+ 147us) service_pool.cc:225] Handling call
0320 15:27:44.866856 (+9231362us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:44.874051 (+ 7195us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:44.949564 (+ 75513us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.149917 (+200353us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.150807 (+ 890us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9231060}
I20260320 15:27:45.151352 30155 leader_election.cc:290] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.151405 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 27ccc693e7704650bd77e0c4ca11862e (table default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]) successfully deleted
W20260320 15:27:45.151692 28383 consensus_peers.cc:597] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6771bd6a3f0e47d99a0191eae25cb2d2. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:45.173049 30456 leader_election.cc:290] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.176759 29640 ts_tablet_manager.cc:1929] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.176929 29640 log.cc:1199] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/906f207dee674a4288c8c60b5d3ff9f8
I20260320 15:27:45.177346 29640 ts_tablet_manager.cc:1950] T 906f207dee674a4288c8c60b5d3ff9f8 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.177726 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 155) took 10951 ms. Trace:
I20260320 15:27:45.177832 29640 rpcz_store.cc:276] 0320 15:27:34.226396 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.226483 (+ 87us) service_pool.cc:225] Handling call
0320 15:27:45.148089 (+10921606us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:45.148104 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.162930 (+ 14826us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.176613 (+ 13683us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.177703 (+ 1090us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10918544}
I20260320 15:27:45.178468 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 906f207dee674a4288c8c60b5d3ff9f8 (table default.Table_5d5d276ee59e4bc68cecddd3cff48146 [id=077c33ae94344e24a25128b4b4a3d92b]) successfully deleted
W20260320 15:27:45.181042 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b1bc86d3d1ff4b3aa8463eb21f619917 because the tablet was not found. No further retry: Not found: Tablet not found: b1bc86d3d1ff4b3aa8463eb21f619917
W20260320 15:27:45.180583 28383 leader_election.cc:343] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.183035 28383 leader_election.cc:304] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:45.183400 29640 tablet_replica.cc:333] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.183580 30344 raft_consensus.cc:2749] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.183678 29640 raft_consensus.cc:2243] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.188943 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:45.191341 28441 raft_consensus.cc:3060] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.206631 28441 raft_consensus.cc:2468] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:45.055873 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 98) took 3631 ms (client timeout 1000 ms). Trace:
I20260320 15:27:45.213661 28524 leader_election.cc:304] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.214357 30155 raft_consensus.cc:2804] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:45.214491 30155 raft_consensus.cc:697] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:45.214779 30155 consensus_queue.cc:237] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.222868 29939 raft_consensus.cc:515] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.227064 28736 tablet_bootstrap.cc:492] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.227265 28736 ts_tablet_manager.cc:1403] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.191s user 0.011s sys 0.049s
I20260320 15:27:45.221246 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 127) took 2692 ms. Trace:
I20260320 15:27:45.228304 28736 raft_consensus.cc:359] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.228511 28736 raft_consensus.cc:385] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.228567 28736 raft_consensus.cc:740] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.228832 28736 consensus_queue.cc:260] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.228013 28905 rpcz_store.cc:276] 0320 15:27:42.528993 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.097467 (+2568474us) service_pool.cc:225] Handling call
0320 15:27:45.114450 (+ 16983us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.117691 (+ 3241us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.118216 (+ 525us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.204541 (+ 86325us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.221019 (+ 16478us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.221163 (+ 144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.126441 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.126583 (+ 142us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.126589 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:45.126592 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.126673 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.126684 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.126689 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.126765 (+ 76us) tablet.cc:700] Row locks acquired
0320 15:27:45.126768 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.126801 (+ 33us) write_op.cc:270] Start()
0320 15:27:45.126836 (+ 35us) write_op.cc:276] Timestamp: P: 1774020465126751 usec, L: 0
0320 15:27:45.126839 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.126991 (+ 152us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.202189 (+ 75198us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.203402 (+ 1213us) write_op.cc:301] APPLY: starting
0320 15:27:45.203417 (+ 15us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.203458 (+ 41us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.203497 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.203626 (+ 129us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.203669 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.203679 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:45.203754 (+ 75us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.203822 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.203840 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:45.203848 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.203935 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.203942 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":38331776,"child_traces":[["op",{"apply.queue_time_us":1170,"apply.run_cpu_time_us":516,"apply.run_wall_time_us":515,"num_ops":2,"prepare.queue_time_us":7709,"prepare.run_cpu_time_us":616,"prepare.run_wall_time_us":615,"replication_time_us":75325}]]}
I20260320 15:27:45.229598 28736 ts_tablet_manager.cc:1434] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.229733 28905 master_service.cc:954] ReplaceTablet: received request to replace tablet fcff253b9cce45b4ad11dc38397afee2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:45.229957 28736 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.221632 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 124) took 2815 ms. Trace:
I20260320 15:27:45.230288 28906 rpcz_store.cc:276] 0320 15:27:42.405709 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.807258 (+2401549us) service_pool.cc:225] Handling call
0320 15:27:44.968946 (+161688us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:44.976731 (+ 7785us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:44.977358 (+ 627us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.186380 (+209022us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.221349 (+ 34969us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.221511 (+ 162us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.064002 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.064219 (+ 217us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.064231 (+ 12us) write_op.cc:435] Acquired schema lock
0320 15:27:45.064236 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:45.064353 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.064366 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.064372 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.064478 (+ 106us) tablet.cc:700] Row locks acquired
0320 15:27:45.064480 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:45.064520 (+ 40us) write_op.cc:270] Start()
0320 15:27:45.064574 (+ 54us) write_op.cc:276] Timestamp: P: 1774020465064475 usec, L: 0
0320 15:27:45.064579 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.064816 (+ 237us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.184359 (+119543us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.184695 (+ 336us) write_op.cc:301] APPLY: starting
0320 15:27:45.184716 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.184883 (+ 167us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.184930 (+ 47us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.185061 (+ 131us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.185103 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.185110 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.185208 (+ 98us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.185285 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.185311 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.185320 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.185404 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.185408 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":339839872,"child_traces":[["op",{"apply.queue_time_us":258,"apply.run_cpu_time_us":626,"apply.run_wall_time_us":702,"num_ops":2,"prepare.queue_time_us":85923,"prepare.run_cpu_time_us":955,"prepare.run_wall_time_us":14786,"replication_time_us":119736}]]}
I20260320 15:27:45.231330 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 4190329df1b644c392ff8cbace9aae7a from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:45.231742 28736 tablet_bootstrap.cc:654] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.221828 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 126) took 2743 ms. Trace:
I20260320 15:27:45.232035 28900 rpcz_store.cc:276] 0320 15:27:42.478255 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.092301 (+2614046us) service_pool.cc:225] Handling call
0320 15:27:45.099093 (+ 6792us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.100150 (+ 1057us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.100810 (+ 660us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.203376 (+102566us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.221687 (+ 18311us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.221765 (+ 78us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.101514 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.101638 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.101645 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:45.101647 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.102460 (+ 813us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.102480 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.102488 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.102592 (+ 104us) tablet.cc:700] Row locks acquired
0320 15:27:45.102595 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.103221 (+ 626us) write_op.cc:270] Start()
0320 15:27:45.103272 (+ 51us) write_op.cc:276] Timestamp: P: 1774020465103164 usec, L: 0
0320 15:27:45.103278 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.126078 (+ 22800us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.126357 (+ 279us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:45.202098 (+ 75741us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.202624 (+ 526us) write_op.cc:301] APPLY: starting
0320 15:27:45.202649 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.202712 (+ 63us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.202717 (+ 5us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.202882 (+ 165us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.202933 (+ 51us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.202945 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:45.203062 (+ 117us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.203153 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.203182 (+ 29us) write_op.cc:454] Released schema lock
0320 15:27:45.203192 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.203299 (+ 107us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.203304 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":14741248,"child_traces":[["op",{"apply.queue_time_us":451,"apply.run_cpu_time_us":664,"apply.run_wall_time_us":663,"num_ops":2,"prepare.queue_time_us":47,"prepare.run_cpu_time_us":1190,"prepare.run_wall_time_us":24883,"replication_time_us":98770,"spinlock_wait_cycles":32356096}]]}
I20260320 15:27:45.222023 28904 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 125) took 2766 ms. Trace:
I20260320 15:27:45.223255 28889 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 128) took 2565 ms. Trace:
I20260320 15:27:45.233639 28889 rpcz_store.cc:276] 0320 15:27:42.657654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.113761 (+2456107us) service_pool.cc:225] Handling call
0320 15:27:45.123577 (+ 9816us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.124177 (+ 600us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.204534 (+ 80357us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.223078 (+ 18544us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.223203 (+ 125us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.127095 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.127186 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.127191 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:45.127194 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.127260 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.127270 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.127274 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.127341 (+ 67us) tablet.cc:700] Row locks acquired
0320 15:27:45.127344 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.127369 (+ 25us) write_op.cc:270] Start()
0320 15:27:45.127394 (+ 25us) write_op.cc:276] Timestamp: P: 1774020465127322 usec, L: 0
0320 15:27:45.127397 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.127557 (+ 160us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.202281 (+ 74724us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.204017 (+ 1736us) write_op.cc:301] APPLY: starting
0320 15:27:45.204029 (+ 12us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.204066 (+ 37us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.204109 (+ 43us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.204219 (+ 110us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.204269 (+ 50us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.204280 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:45.204346 (+ 66us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.204395 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.204412 (+ 17us) write_op.cc:454] Released schema lock
0320 15:27:45.204419 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1695,"apply.run_cpu_time_us":472,"apply.run_wall_time_us":472,"num_ops":2,"prepare.queue_time_us":2176,"prepare.run_cpu_time_us":520,"prepare.run_wall_time_us":519,"replication_time_us":74853,"thread_start_us":27634,"threads_started":1}]]}
I20260320 15:27:45.233631 28904 rpcz_store.cc:276] 0320 15:27:42.455891 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.827754 (+2371863us) service_pool.cc:225] Handling call
0320 15:27:45.006892 (+179138us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.021739 (+ 14847us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.033772 (+ 12033us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.188041 (+154269us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.221894 (+ 33853us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.221975 (+ 81us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.078840 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.078996 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.079004 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:45.079006 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.079110 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.079125 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.079131 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.079229 (+ 98us) tablet.cc:700] Row locks acquired
0320 15:27:45.079234 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:45.079286 (+ 52us) write_op.cc:270] Start()
0320 15:27:45.079336 (+ 50us) write_op.cc:276] Timestamp: P: 1774020465079238 usec, L: 0
0320 15:27:45.079341 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.079638 (+ 297us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.184455 (+104817us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.185488 (+ 1033us) write_op.cc:301] APPLY: starting
0320 15:27:45.185500 (+ 12us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.185555 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.185594 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.185751 (+ 157us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.185786 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.185793 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.185879 (+ 86us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.185933 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.185957 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:45.185966 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.186045 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.186048 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":411012352,"child_traces":[["op",{"apply.queue_time_us":995,"apply.run_cpu_time_us":548,"apply.run_wall_time_us":547,"num_ops":2,"prepare.queue_time_us":44478,"prepare.run_cpu_time_us":920,"prepare.run_wall_time_us":3504,"replication_time_us":105094}]]}
W20260320 15:27:45.215821 28901 rpcz_store.cc:269] 0320 15:27:41.424765 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.484196 (+ 59431us) service_pool.cc:225] Handling call
0320 15:27:42.673444 (+1189248us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.673452 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:45.055818 (+2382366us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:42.665659 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:42.665785 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:42.665793 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:42.665796 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:42.665925 (+ 129us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:42.665941 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:42.665946 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:42.666060 (+ 114us) tablet.cc:700] Row locks acquired
0320 15:27:42.666064 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:42.666101 (+ 37us) write_op.cc:270] Start()
0320 15:27:42.666136 (+ 35us) write_op.cc:276] Timestamp: P: 1774020462666039 usec, L: 0
0320 15:27:42.666140 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:42.666345 (+ 205us) log.cc:844] Serialized 901 byte log entry
0320 15:27:42.669980 (+ 3635us) op_driver.cc:464] REPLICATION: finished
0320 15:27:42.670107 (+ 127us) write_op.cc:301] APPLY: starting
0320 15:27:42.670127 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:42.670194 (+ 67us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:42.670229 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:42.670318 (+ 89us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:42.670359 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:42.670367 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:42.670448 (+ 81us) log.cc:844] Serialized 29 byte log entry
0320 15:27:42.670517 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:42.670542 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:42.670550 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":213640,"reported_tablets":200,"child_traces":[["op",{"apply.queue_time_us":74,"apply.run_cpu_time_us":506,"apply.run_wall_time_us":505,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":793,"prepare.run_wall_time_us":794,"raft.queue_time_us":33,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":254,"replication_time_us":3800}]]}
I20260320 15:27:45.242147 28736 tablet_bootstrap.cc:492] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.242980 28736 ts_tablet_manager.cc:1403] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260320 15:27:45.245023 28736 raft_consensus.cc:359] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.245234 28736 raft_consensus.cc:385] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.245307 28736 raft_consensus.cc:740] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.245525 28736 consensus_queue.cc:260] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.246045 28736 ts_tablet_manager.cc:1434] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260320 15:27:45.246361 28736 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.252868 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 3d985575b862410792b016c53df4d139 (DEFAULT_TABLE table=default.Table_59eba3ec25134b28aeb775da692dae4b [id=268b3f3861da4c5c9e1691f87a0ea784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.253119 28413 tablet_service.cc:1511] Processing CreateTablet for tablet 3226f3fe64c14ec6b0ff01fe5a99add2 (DEFAULT_TABLE table=default.Table_6a46b3258ceb4784a9b3ed584b7fcae1 [id=5b2a9b0165384ee2946a512198267872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.253291 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d985575b862410792b016c53df4d139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.253494 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226f3fe64c14ec6b0ff01fe5a99add2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.255962 28736 tablet_bootstrap.cc:654] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.252868 28425 tablet_service.cc:1511] Processing CreateTablet for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (DEFAULT_TABLE table=default.Table_76bddc5536ff43e895889c5c30c5068f [id=7be232d8d79846eb858e6dd529427db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.256364 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ac11b0ca134c948fdd9ff4b114e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.275197 28736 tablet_bootstrap.cc:492] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.276098 28736 ts_tablet_manager.cc:1403] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.002s
W20260320 15:27:45.227612 28524 leader_election.cc:343] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.250636 28555 tablet_service.cc:1511] Processing CreateTablet for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (DEFAULT_TABLE table=default.Table_76bddc5536ff43e895889c5c30c5068f [id=7be232d8d79846eb858e6dd529427db4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:45.278678 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 51095us
I20260320 15:27:45.278829 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ac11b0ca134c948fdd9ff4b114e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:45.278921 28524 leader_election.cc:343] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:45.279186 28524 leader_election.cc:343] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:45.279435 28524 leader_election.cc:343] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:45.280311 28524 leader_election.cc:343] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.280990 28524 leader_election.cc:304] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:45.280385 28736 raft_consensus.cc:359] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.281287 30394 raft_consensus.cc:3060] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.281332 28736 raft_consensus.cc:385] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.281677 28736 raft_consensus.cc:740] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.251245 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 3226f3fe64c14ec6b0ff01fe5a99add2 (DEFAULT_TABLE table=default.Table_6a46b3258ceb4784a9b3ed584b7fcae1 [id=5b2a9b0165384ee2946a512198267872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.251998 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 3d985575b862410792b016c53df4d139 (DEFAULT_TABLE table=default.Table_59eba3ec25134b28aeb775da692dae4b [id=268b3f3861da4c5c9e1691f87a0ea784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.254130 29640 raft_consensus.cc:2272] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.283022 28736 consensus_queue.cc:260] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:45.281955 28524 leader_election.cc:343] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.282476 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226f3fe64c14ec6b0ff01fe5a99add2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.284162 29640 ts_tablet_manager.cc:1916] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.285080 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d985575b862410792b016c53df4d139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.285610 30355 consensus_queue.cc:1048] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.286597 28736 ts_tablet_manager.cc:1434] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260320 15:27:45.288425 28736 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.247772 28903 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 129) took 2560 ms. Trace:
I20260320 15:27:45.293524 28903 rpcz_store.cc:276] 0320 15:27:42.686957 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.115824 (+2428867us) service_pool.cc:225] Handling call
0320 15:27:45.124547 (+ 8723us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.125330 (+ 783us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.247317 (+121987us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.247581 (+ 264us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.247700 (+ 119us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.127675 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.127760 (+ 85us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.127765 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:45.127767 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.127835 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.127843 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.127847 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.127935 (+ 88us) tablet.cc:700] Row locks acquired
0320 15:27:45.127938 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.127966 (+ 28us) write_op.cc:270] Start()
0320 15:27:45.127991 (+ 25us) write_op.cc:276] Timestamp: P: 1774020465127917 usec, L: 0
0320 15:27:45.127994 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.128127 (+ 133us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.230127 (+102000us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.246307 (+ 16180us) write_op.cc:301] APPLY: starting
0320 15:27:45.246332 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.246391 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.246469 (+ 78us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.246676 (+ 207us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.246715 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.246723 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:45.246826 (+ 103us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.247176 (+ 350us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.247203 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:45.247214 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.247306 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.247310 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":16065,"apply.run_cpu_time_us":775,"apply.run_wall_time_us":1006,"num_ops":2,"prepare.queue_time_us":1735,"prepare.run_cpu_time_us":510,"prepare.run_wall_time_us":507,"replication_time_us":102097}]]}
I20260320 15:27:45.294507 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_13967321267549daa281ef67538d1894" } new_table_name: "default.Table_20b641db350b494dbd1c458752f5a7b9" modify_external_catalogs: true
I20260320 15:27:45.253360 28900 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_7abb6e0bedbe439daf8e7bbede311b1a" } new_table_name: "default.Table_c78295faeae84addb0a4a445fef9bfbd" modify_external_catalogs: true
I20260320 15:27:45.256366 28889 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_9b2e32517f9d49a9a117be78ffddb91e" } modify_external_catalogs: true
W20260320 15:27:45.278311 28870 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 71db28879e554f578b8a7b0e14419ab7 consensus state has no leader
I20260320 15:27:45.295987 30454 raft_consensus.cc:493] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.296337 30454 raft_consensus.cc:515] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.298758 28736 tablet_bootstrap.cc:654] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.299863 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.301234 28382 leader_election.cc:343] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.306083 30355 raft_consensus.cc:493] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.306239 30355 raft_consensus.cc:515] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.306859 30355 leader_election.cc:290] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.314990 30394 raft_consensus.cc:2749] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.317870 29643 tablet_replica.cc:333] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.318218 29643 raft_consensus.cc:2243] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.318711 29643 raft_consensus.cc:2272] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.318745 28733 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.319061 28733 ts_tablet_manager.cc:1403] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.297s user 0.022s sys 0.029s
I20260320 15:27:45.319190 28736 tablet_bootstrap.cc:492] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.320528 28733 raft_consensus.cc:359] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.320783 28736 ts_tablet_manager.cc:1403] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.004s
I20260320 15:27:45.320871 28733 raft_consensus.cc:385] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.320994 28733 raft_consensus.cc:740] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.321751 29643 ts_tablet_manager.cc:1916] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.321415 28733 consensus_queue.cc:260] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.323166 28902 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 130) took 2594 ms. Trace:
I20260320 15:27:45.323216 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:45.324025 28902 rpcz_store.cc:276] 0320 15:27:42.729102 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.143498 (+2414396us) service_pool.cc:225] Handling call
0320 15:27:45.220656 (+ 77158us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:45.221366 (+ 710us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:45.268570 (+ 47204us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:45.322859 (+ 54289us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:45.323074 (+ 215us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.222265 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.222454 (+ 189us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.222465 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:45.222470 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:45.222596 (+ 126us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.222614 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.222622 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.222737 (+ 115us) tablet.cc:700] Row locks acquired
0320 15:27:45.222742 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:45.222792 (+ 50us) write_op.cc:270] Start()
0320 15:27:45.222841 (+ 49us) write_op.cc:276] Timestamp: P: 1774020465222728 usec, L: 0
0320 15:27:45.222847 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.260070 (+ 37223us) log.cc:844] Serialized 433 byte log entry
0320 15:27:45.260306 (+ 236us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:45.265811 (+ 5505us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.267058 (+ 1247us) write_op.cc:301] APPLY: starting
0320 15:27:45.267202 (+ 144us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.267268 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.267273 (+ 5us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.267900 (+ 627us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.267952 (+ 52us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:45.267971 (+ 19us) write_op.cc:312] APPLY: finished
0320 15:27:45.268092 (+ 121us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.268427 (+ 335us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.268455 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:45.268466 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.268571 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.268575 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1145,"apply.run_cpu_time_us":941,"apply.run_wall_time_us":1517,"num_ops":2,"prepare.queue_time_us":73,"prepare.run_cpu_time_us":1790,"prepare.run_wall_time_us":38126,"raft.queue_time_us":84,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":1048,"replication_time_us":42924,"spinlock_wait_cycles":85096704}]]}
I20260320 15:27:45.324810 28902 master_service.cc:954] ReplaceTablet: received request to replace tablet 41da3cb06b1147fbae2695bc5c9ffaa5 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:45.309839 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_047ea6293f154736b2bff4875bc8cd43" } new_table_name: "default.Table_929744d40f1a48c5aa6e0d16d97e93c3" modify_external_catalogs: true
I20260320 15:27:45.325528 29678 ts_tablet_manager.cc:1929] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:45.325657 29678 log.cc:1199] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:45.325927 28889 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_9b2e32517f9d49a9a117be78ffddb91e" } modify_external_catalogs: true
I20260320 15:27:45.326112 29678 ts_tablet_manager.cc:1950] T 5a20d7f22cfa403184e73b24d565b60a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.326778 29640 ts_tablet_manager.cc:1929] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.326889 29640 log.cc:1199] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7f0242a7968541f480367b45cda5d9cc
I20260320 15:27:45.322217 28734 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.327247 29640 ts_tablet_manager.cc:1950] T 7f0242a7968541f480367b45cda5d9cc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.327288 28734 ts_tablet_manager.cc:1403] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.275s user 0.025s sys 0.057s
I20260320 15:27:45.327942 28736 raft_consensus.cc:359] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.328119 28736 raft_consensus.cc:385] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.328173 28736 raft_consensus.cc:740] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.328101 28734 raft_consensus.cc:359] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.328258 28734 raft_consensus.cc:385] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.328331 28734 raft_consensus.cc:740] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.328527 28734 consensus_queue.cc:260] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.328923 28736 consensus_queue.cc:260] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.329291 28734 ts_tablet_manager.cc:1434] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.329407 28736 ts_tablet_manager.cc:1434] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260320 15:27:45.329684 28736 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.331842 28734 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.332713 28736 tablet_bootstrap.cc:654] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.336365 28905 catalog_manager.cc:6741] ReplaceTablet: tablet fcff253b9cce45b4ad11dc38397afee2 deleted and replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73
I20260320 15:27:45.336586 28905 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 131) took 2449 ms. Trace:
I20260320 15:27:45.336681 28905 rpcz_store.cc:276] 0320 15:27:42.886847 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.229554 (+2342707us) service_pool.cc:225] Handling call
0320 15:27:45.267777 (+ 38223us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.336488 (+ 68711us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.274424 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.274616 (+ 192us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.274626 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:45.274631 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:45.274820 (+ 189us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.274841 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.274849 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.274934 (+ 85us) tablet.cc:700] Row locks acquired
0320 15:27:45.274937 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.274972 (+ 35us) write_op.cc:270] Start()
0320 15:27:45.275013 (+ 41us) write_op.cc:276] Timestamp: P: 1774020465274956 usec, L: 0
0320 15:27:45.275019 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.275328 (+ 309us) log.cc:844] Serialized 656 byte log entry
0320 15:27:45.290307 (+ 14979us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.290458 (+ 151us) write_op.cc:301] APPLY: starting
0320 15:27:45.290489 (+ 31us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.290545 (+ 56us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.290649 (+ 104us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.290787 (+ 138us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.290847 (+ 60us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.290854 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.291098 (+ 244us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.291411 (+ 313us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.291442 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:45.291448 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.291710 (+ 262us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.291717 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":86615936,"child_traces":[["op",{"apply.queue_time_us":52,"apply.run_cpu_time_us":709,"apply.run_wall_time_us":1074,"num_ops":2,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":1051,"prepare.run_wall_time_us":1050,"raft.queue_time_us":451,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":5547,"replication_time_us":15235}]]}
I20260320 15:27:45.344152 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet fcff253b9cce45b4ad11dc38397afee2 with delete_type TABLET_DATA_DELETED (replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:45.344691 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:45.321972 30454 leader_election.cc:290] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.345029 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 199) took 8260 ms. Trace:
I20260320 15:27:45.345129 29678 rpcz_store.cc:276] 0320 15:27:37.084500 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.085011 (+ 511us) service_pool.cc:225] Handling call
0320 15:27:44.911625 (+7826614us) ts_tablet_manager.cc:1151] Deleting tablet 5a20d7f22cfa403184e73b24d565b60a
0320 15:27:44.911634 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.031237 (+119603us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.325392 (+294155us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.344991 (+ 19599us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7826477}
I20260320 15:27:45.345680 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 201) took 8202 ms. Trace:
I20260320 15:27:45.345767 29678 rpcz_store.cc:276] 0320 15:27:37.143143 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.143327 (+ 184us) service_pool.cc:225] Handling call
0320 15:27:45.345553 (+8202226us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:45.345566 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.345664 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8178989}
I20260320 15:27:45.346105 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 200) took 8203 ms. Trace:
I20260320 15:27:45.346184 29678 rpcz_store.cc:276] 0320 15:27:37.143040 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.143118 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:45.345997 (+8202879us) ts_tablet_manager.cc:1151] Deleting tablet 197f1935099140a98c5eb801cf44046a
0320 15:27:45.346006 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.346091 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8179240}
I20260320 15:27:45.346477 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 202) took 8090 ms. Trace:
I20260320 15:27:45.346542 29678 rpcz_store.cc:276] 0320 15:27:37.256526 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.256604 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:45.346404 (+8089800us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:45.346412 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.346462 (+ 50us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8089603}
I20260320 15:27:45.346820 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 204) took 6805 ms. Trace:
I20260320 15:27:45.346881 29678 rpcz_store.cc:276] 0320 15:27:38.541640 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.541751 (+ 111us) service_pool.cc:225] Handling call
0320 15:27:45.346717 (+6804966us) ts_tablet_manager.cc:1151] Deleting tablet 20eec704800e4d8b924d97556cc2700e
0320 15:27:45.346727 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.346804 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6804790}
I20260320 15:27:45.347124 29678 tablet_replica.cc:333] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.347343 29678 raft_consensus.cc:2243] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.347481 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 156) took 11114 ms. Trace:
I20260320 15:27:45.341082 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet fcff253b9cce45b4ad11dc38397afee2 with delete_type TABLET_DATA_DELETED (replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.327713 28733 ts_tablet_manager.cc:1434] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260320 15:27:45.347908 29640 rpcz_store.cc:276] 0320 15:27:34.232986 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.233662 (+ 676us) service_pool.cc:225] Handling call
0320 15:27:45.183326 (+10949664us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:45.183340 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.298313 (+114973us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.326661 (+ 28348us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.347462 (+ 20801us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10946295}
I20260320 15:27:45.348687 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 158) took 11102 ms. Trace:
I20260320 15:27:45.348783 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7f0242a7968541f480367b45cda5d9cc (table default.Table_f9e940b8c37f46a6a671a47344d1c74d [id=d9986627021e48c3be0c623ca3fcf822]) successfully deleted
I20260320 15:27:45.348814 29640 rpcz_store.cc:276] 0320 15:27:34.245951 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.248208 (+ 2257us) service_pool.cc:225] Handling call
0320 15:27:45.348598 (+11100390us) ts_tablet_manager.cc:1151] Deleting tablet 825f7e3fdc004fe1a3bcca6deb20cccd
0320 15:27:45.348607 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.348673 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11100109}
W20260320 15:27:45.349228 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 825f7e3fdc004fe1a3bcca6deb20cccd because the tablet was not found. No further retry: Not found: Tablet not found: 825f7e3fdc004fe1a3bcca6deb20cccd
I20260320 15:27:45.348212 28733 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.351688 28733 tablet_bootstrap.cc:654] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:45.351831 28892 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 109) took 1388 ms (client timeout 1000 ms). Trace:
I20260320 15:27:45.352849 29640 tablet_replica.cc:333] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.353142 29640 raft_consensus.cc:2243] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.355361 29640 raft_consensus.cc:2272] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:45.355741 28892 rpcz_store.cc:269] 0320 15:27:43.963189 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.014905 (+ 51716us) service_pool.cc:225] Handling call
0320 15:27:45.351777 (+1336872us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":55881,"reported_tablets":179}
I20260320 15:27:45.356063 29640 ts_tablet_manager.cc:1916] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.357412 28892 master_service.cc:954] ReplaceTablet: received request to replace tablet cc97a5f2bd964a05bb7b2427226debdd from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:45.357846 29678 raft_consensus.cc:2272] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.359769 29678 ts_tablet_manager.cc:1916] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.361887 28736 tablet_bootstrap.cc:492] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.362020 28736 ts_tablet_manager.cc:1403] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20260320 15:27:45.362423 29640 ts_tablet_manager.cc:1929] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:45.362871 28736 raft_consensus.cc:359] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.363039 28736 raft_consensus.cc:385] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.363087 28736 raft_consensus.cc:740] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.364221 28736 consensus_queue.cc:260] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.365492 28736 ts_tablet_manager.cc:1434] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:45.365787 28736 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.371850 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:45.373056 28892 catalog_manager.cc:6741] ReplaceTablet: tablet cc97a5f2bd964a05bb7b2427226debdd deleted and replaced by tablet 9a2003e2af9a4d9c854990e1db41af28
I20260320 15:27:45.374020 28892 master_service.cc:954] ReplaceTablet: received request to replace tablet d04ab44039124f36ac60601098586b4e from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:45.374652 28736 tablet_bootstrap.cc:654] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.375674 29640 log.cc:1199] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/0c2339ff2c474a288876f32b85cecc72
I20260320 15:27:45.375726 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet cc97a5f2bd964a05bb7b2427226debdd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a2003e2af9a4d9c854990e1db41af28) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.376564 28733 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.376901 28733 ts_tablet_manager.cc:1403] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.003s
I20260320 15:27:45.377836 28733 raft_consensus.cc:359] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.378079 28733 raft_consensus.cc:385] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.378181 28733 raft_consensus.cc:740] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.378316 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:45.378480 29678 ts_tablet_manager.cc:1929] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.378434 28733 consensus_queue.cc:260] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.378607 29678 log.cc:1199] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:45.378988 29678 ts_tablet_manager.cc:1950] T fa5618796f1a4ff3ab4c1209dad0aede P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.379021 28733 ts_tablet_manager.cc:1434] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.379304 28733 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.379803 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 205) took 6091 ms. Trace:
I20260320 15:27:45.379901 29678 rpcz_store.cc:276] 0320 15:27:39.288155 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.288239 (+ 84us) service_pool.cc:225] Handling call
0320 15:27:45.347080 (+6058841us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:45.347089 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.366769 (+ 19680us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.378362 (+ 11593us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.379780 (+ 1418us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6058557}
I20260320 15:27:45.380288 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 207) took 6091 ms. Trace:
I20260320 15:27:45.380369 29678 rpcz_store.cc:276] 0320 15:27:39.288431 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.288587 (+ 156us) service_pool.cc:225] Handling call
0320 15:27:45.380176 (+6091589us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:45.380185 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.380264 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6091411}
I20260320 15:27:45.380654 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 209) took 6092 ms. Trace:
I20260320 15:27:45.380726 29678 rpcz_store.cc:276] 0320 15:27:39.288655 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.288720 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:45.380581 (+6091861us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:45.380590 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.380641 (+ 51us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6091674}
I20260320 15:27:45.380967 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 210) took 6092 ms. Trace:
I20260320 15:27:45.381034 29678 rpcz_store.cc:276] 0320 15:27:39.288751 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.288902 (+ 151us) service_pool.cc:225] Handling call
0320 15:27:45.380905 (+6092003us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:45.380912 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.380954 (+ 42us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6091853}
I20260320 15:27:45.381139 28733 tablet_bootstrap.cc:654] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.381273 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 206) took 6092 ms. Trace:
I20260320 15:27:45.381350 29678 rpcz_store.cc:276] 0320 15:27:39.288317 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.288587 (+ 270us) service_pool.cc:225] Handling call
0320 15:27:45.381210 (+6092623us) ts_tablet_manager.cc:1151] Deleting tablet fa22d207ad8a4555aa9573b15fa1fe1a
0320 15:27:45.381218 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.381261 (+ 43us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6092009}
I20260320 15:27:45.381587 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 211) took 6092 ms. Trace:
I20260320 15:27:45.381655 29678 rpcz_store.cc:276] 0320 15:27:39.288840 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.289244 (+ 404us) service_pool.cc:225] Handling call
0320 15:27:45.381527 (+6092283us) ts_tablet_manager.cc:1151] Deleting tablet bcec6175c1ca49f5b682a29d5c1e235a
0320 15:27:45.381534 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.381575 (+ 41us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6092143}
I20260320 15:27:45.381891 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 208) took 6093 ms. Trace:
I20260320 15:27:45.381953 29678 rpcz_store.cc:276] 0320 15:27:39.288556 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.289450 (+ 894us) service_pool.cc:225] Handling call
0320 15:27:45.381832 (+6092382us) ts_tablet_manager.cc:1151] Deleting tablet e2586d40343b4365ab8a67cce2e1c6ca
0320 15:27:45.381839 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.381879 (+ 40us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6092210}
I20260320 15:27:45.382167 29678 tablet_replica.cc:333] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.382356 29678 raft_consensus.cc:2243] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.382513 29678 raft_consensus.cc:2272] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.382521 29640 ts_tablet_manager.cc:1950] T 0c2339ff2c474a288876f32b85cecc72 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.383033 29678 ts_tablet_manager.cc:1916] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.383263 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 159) took 11137 ms. Trace:
I20260320 15:27:45.383399 29640 rpcz_store.cc:276] 0320 15:27:34.246101 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.253702 (+ 7601us) service_pool.cc:225] Handling call
0320 15:27:45.352797 (+11099095us) ts_tablet_manager.cc:1151] Deleting tablet 0c2339ff2c474a288876f32b85cecc72
0320 15:27:45.352807 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.359360 (+ 6553us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.362322 (+ 2962us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.383240 (+ 20918us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11098840}
I20260320 15:27:45.383826 29640 tablet_replica.cc:333] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.384078 29640 raft_consensus.cc:2243] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.387259 29640 raft_consensus.cc:2272] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.387283 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:45.387883 28890 catalog_manager.cc:5671] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:45.388495 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:45.388598 29640 ts_tablet_manager.cc:1916] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.388967 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:45.389173 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:45.389847 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:45.389981 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 0c2339ff2c474a288876f32b85cecc72 (table default.Table_d4066b167fcb4656b497ac2ef62d5366 [id=cbccbfa505024b1b89d6c80ef592a812]) successfully deleted
I20260320 15:27:45.390244 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:45.390499 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:45.390771 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:45.396692 28733 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.396946 28733 ts_tablet_manager.cc:1403] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260320 15:27:45.397735 28733 raft_consensus.cc:359] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.398011 28733 raft_consensus.cc:385] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.398105 28733 raft_consensus.cc:740] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.398389 28733 consensus_queue.cc:260] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.398963 28733 ts_tablet_manager.cc:1434] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.347837 28734 tablet_bootstrap.cc:654] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.365453 29643 ts_tablet_manager.cc:1929] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.399191 29643 log.cc:1199] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6771bd6a3f0e47d99a0191eae25cb2d2
I20260320 15:27:45.399767 29643 ts_tablet_manager.cc:1950] T 6771bd6a3f0e47d99a0191eae25cb2d2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:45.400743 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 182) took 9653 ms. Trace:
I20260320 15:27:45.400851 29643 rpcz_store.cc:276] 0320 15:27:35.747095 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.761230 (+ 14135us) service_pool.cc:225] Handling call
0320 15:27:45.317804 (+9556574us) ts_tablet_manager.cc:1151] Deleting tablet 6771bd6a3f0e47d99a0191eae25cb2d2
0320 15:27:45.317819 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.332510 (+ 14691us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.365325 (+ 32815us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.400718 (+ 35393us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9556299}
I20260320 15:27:45.401355 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 191) took 8406 ms. Trace:
I20260320 15:27:45.401465 29643 rpcz_store.cc:276] 0320 15:27:36.995155 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.995587 (+ 432us) service_pool.cc:225] Handling call
0320 15:27:45.401161 (+8405574us) ts_tablet_manager.cc:1151] Deleting tablet 7f94140a5e8b4a06890bca86456d64a6
0320 15:27:45.401175 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.401332 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8405337}
I20260320 15:27:45.401858 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 196) took 8406 ms. Trace:
I20260320 15:27:45.401949 29643 rpcz_store.cc:276] 0320 15:27:36.995684 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.995763 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:45.401736 (+8405973us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:45.401748 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.401840 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8405748}
I20260320 15:27:45.402272 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 197) took 8406 ms. Trace:
I20260320 15:27:45.402377 29643 rpcz_store.cc:276] 0320 15:27:36.995770 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.995833 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:45.402192 (+8406359us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:45.402203 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.402260 (+ 57us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24,"tablet-delete.queue_time_us":8406203}
I20260320 15:27:45.402774 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 192) took 8407 ms. Trace:
I20260320 15:27:45.390635 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet cc97a5f2bd964a05bb7b2427226debdd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a2003e2af9a4d9c854990e1db41af28) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:45.403614 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (table default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]) successfully deleted
W20260320 15:27:45.404007 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
I20260320 15:27:45.404244 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
W20260320 15:27:45.404253 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet edb7b53bb7444a39b471218c6039542f because the tablet was not found. No further retry: Not found: Tablet not found: edb7b53bb7444a39b471218c6039542f
W20260320 15:27:45.404789 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet a2775196c8e04dcc80c7e089b57579fd because the tablet was not found. No further retry: Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:45.404858 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 23eda54577404cc6900cb3ae348604d4 (table default.Table_ab12be810262486ca2ec3234763b510b [id=bf25edfb9f0e4924aadf3e85e57fab77]). Expected version 1 got 0
I20260320 15:27:45.402865 29643 rpcz_store.cc:276] 0320 15:27:36.995254 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.995854 (+ 600us) service_pool.cc:225] Handling call
0320 15:27:45.402669 (+8406815us) ts_tablet_manager.cc:1151] Deleting tablet 9aa2d48a54dd403c82d7f3a64db5aeab
0320 15:27:45.402681 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.402756 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8406618}
I20260320 15:27:45.405213 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
W20260320 15:27:45.405570 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because the tablet was not found. No further retry: Not found: Tablet not found: 9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:45.406107 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 201) took 8409 ms. Trace:
I20260320 15:27:45.406242 29643 rpcz_store.cc:276] 0320 15:27:36.996147 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.996202 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:45.405972 (+8409770us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:45.405989 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.406082 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8409482}
I20260320 15:27:45.406288 28901 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_e9d83c80d8c64b19af48bda73dff8d8c" } modify_external_catalogs: true
I20260320 15:27:45.407115 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 205) took 8410 ms. Trace:
I20260320 15:27:45.407181 29640 ts_tablet_manager.cc:1929] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:45.407200 29643 rpcz_store.cc:276] 0320 15:27:36.996611 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.996669 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:45.407020 (+8410351us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:45.407030 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.407095 (+ 65us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8410180}
I20260320 15:27:45.407371 29640 log.cc:1199] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/d547c032332d4d109222b4743fa28561
I20260320 15:27:45.407691 29643 tablet_replica.cc:333] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.407733 29640 ts_tablet_manager.cc:1950] T d547c032332d4d109222b4743fa28561 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.407912 29643 raft_consensus.cc:2243] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.408067 29643 raft_consensus.cc:2272] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.408648 29643 ts_tablet_manager.cc:1916] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:45.408849 28239 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.27.74.60:42563, user_credentials={real_user=slave}} blocked reactor thread for 80263.2us
I20260320 15:27:45.409225 29939 leader_election.cc:290] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:45.406857 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 993e2fb1561f4317be85e9b5e5816edc because the tablet was not found. No further retry: Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:45.407040 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:45.411520 28736 tablet_bootstrap.cc:492] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.411691 28736 ts_tablet_manager.cc:1403] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.046s user 0.000s sys 0.004s
I20260320 15:27:45.411674 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:45.412143 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:45.412798 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:45.413069 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:45.415277 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.415500 28440 raft_consensus.cc:2468] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:45.422775 29643 ts_tablet_manager.cc:1929] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.422933 29643 log.cc:1199] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:45.423327 29643 ts_tablet_manager.cc:1950] T 7a4b5a658cd84b04b50b54973c065889 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:45.409668 28733 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.413964 28901 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_e9d83c80d8c64b19af48bda73dff8d8c" } modify_external_catalogs: true
I20260320 15:27:45.413728 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:45.429584 29678 ts_tablet_manager.cc:1929] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:45.429572 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:45.429714 29678 log.cc:1199] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:45.429880 28733 tablet_bootstrap.cc:654] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.430040 29678 ts_tablet_manager.cc:1950] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.430390 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:45.410290 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 160) took 11164 ms. Trace:
I20260320 15:27:45.430999 29640 rpcz_store.cc:276] 0320 15:27:34.246191 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.253997 (+ 7806us) service_pool.cc:225] Handling call
0320 15:27:45.383781 (+11129784us) ts_tablet_manager.cc:1151] Deleting tablet d547c032332d4d109222b4743fa28561
0320 15:27:45.383790 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.393502 (+ 9712us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.407057 (+ 13555us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.410260 (+ 3203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11129614}
I20260320 15:27:45.431392 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:45.431455 29640 tablet_replica.cc:333] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.431669 29640 raft_consensus.cc:2243] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.431936 29640 raft_consensus.cc:2272] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.432248 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:45.432518 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:45.432579 29640 ts_tablet_manager.cc:1916] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.432796 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:45.433202 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:45.433471 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:45.433688 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:45.434091 28890 catalog_manager.cc:5671] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:45.434552 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:45.434880 28890 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:45.436211 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet d547c032332d4d109222b4743fa28561 (table default.Table_bb5eb65d708a4ece947d11468e68b8d4 [id=c4809856807c41debfddd0116dc7aeb8]) successfully deleted
I20260320 15:27:45.433609 28734 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.436435 28734 ts_tablet_manager.cc:1403] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.105s user 0.008s sys 0.001s
I20260320 15:27:45.437263 28734 raft_consensus.cc:359] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.437439 28734 raft_consensus.cc:385] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.437510 28734 raft_consensus.cc:740] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.437736 28734 consensus_queue.cc:260] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.438485 28734 ts_tablet_manager.cc:1434] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.443466 28734 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.449191 29640 ts_tablet_manager.cc:1929] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.449563 28734 tablet_bootstrap.cc:654] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.449837 29640 log.cc:1199] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6292904782fd4b619483135e58b66684
I20260320 15:27:45.450374 29640 ts_tablet_manager.cc:1950] T 6292904782fd4b619483135e58b66684 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.450714 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 198) took 8454 ms. Trace:
I20260320 15:27:45.450816 29643 rpcz_store.cc:276] 0320 15:27:36.995872 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.996823 (+ 951us) service_pool.cc:225] Handling call
0320 15:27:45.407631 (+8410808us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:45.407640 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.412590 (+ 4950us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.422640 (+ 10050us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.450692 (+ 28052us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8410617}
I20260320 15:27:45.451218 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 206) took 8454 ms. Trace:
I20260320 15:27:45.451308 29643 rpcz_store.cc:276] 0320 15:27:36.996892 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.996949 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:45.451106 (+8454157us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:45.451114 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.451190 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8453993}
I20260320 15:27:45.451598 29643 tablet_replica.cc:333] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.451803 29643 raft_consensus.cc:2243] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.451972 29643 raft_consensus.cc:2272] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.452544 29643 ts_tablet_manager.cc:1916] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.453310 30491 mvcc.cc:204] Tried to move back new op lower bound from 7266387778492170240 to 7266387757337276416. Current Snapshot: MvccSnapshot[applied={T|T < 7266387778492170240 or (T in {7266387778492170240})}]
I20260320 15:27:45.437952 28736 raft_consensus.cc:359] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.455643 28736 raft_consensus.cc:385] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.455729 28736 raft_consensus.cc:740] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.455945 28736 consensus_queue.cc:260] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.456480 28736 ts_tablet_manager.cc:1434] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.045s user 0.000s sys 0.001s
I20260320 15:27:45.456775 28736 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.458544 28736 tablet_bootstrap.cc:654] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.458787 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 161) took 11212 ms. Trace:
I20260320 15:27:45.461766 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 212) took 6066 ms. Trace:
I20260320 15:27:45.461891 29678 rpcz_store.cc:276] 0320 15:27:39.395381 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.395487 (+ 106us) service_pool.cc:225] Handling call
0320 15:27:45.382124 (+5986637us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:45.382131 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.426274 (+ 44143us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.429460 (+ 3186us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.461744 (+ 32284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5907909}
I20260320 15:27:45.462325 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 213) took 6066 ms. Trace:
I20260320 15:27:45.462544 29678 rpcz_store.cc:276] 0320 15:27:39.395843 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.395956 (+ 113us) service_pool.cc:225] Handling call
0320 15:27:45.462199 (+6066243us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:45.462221 (+ 22us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.462306 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5987889}
I20260320 15:27:45.462913 28736 tablet_bootstrap.cc:492] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.463282 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 7a4b5a658cd84b04b50b54973c065889 (table default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]) successfully deleted
I20260320 15:27:45.463210 29640 rpcz_store.cc:276] 0320 15:27:34.246267 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.254204 (+ 7937us) service_pool.cc:225] Handling call
0320 15:27:45.431403 (+11177199us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:45.431413 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.438818 (+ 7405us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.449056 (+ 10238us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.458755 (+ 9699us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11177020}
I20260320 15:27:45.463564 30344 raft_consensus.cc:493] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:45.463712 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 931f6f88a1d94a40b69f07216b8313c8 because the tablet was not found. No further retry: Not found: Tablet not found: 931f6f88a1d94a40b69f07216b8313c8
I20260320 15:27:45.463711 30344 raft_consensus.cc:515] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.463850 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 6292904782fd4b619483135e58b66684 (table default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]) successfully deleted
I20260320 15:27:45.464376 30344 leader_election.cc:290] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.464797 29640 tablet_replica.cc:333] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.464847 28736 ts_tablet_manager.cc:1403] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260320 15:27:45.465040 29640 raft_consensus.cc:2243] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.464797 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e9791ac61b34fc09a8292c79f0c4032" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.465394 29640 raft_consensus.cc:2272] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:45.465621 28382 leader_election.cc:343] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.465750 28736 raft_consensus.cc:359] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.465914 28736 raft_consensus.cc:385] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.465957 28736 raft_consensus.cc:740] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.465999 29640 ts_tablet_manager.cc:1916] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.466150 28736 consensus_queue.cc:260] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.439662 28239 leader_election.cc:304] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:45.466670 28736 ts_tablet_manager.cc:1434] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:45.466771 30466 raft_consensus.cc:3060] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.466951 28736 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.468755 28736 tablet_bootstrap.cc:654] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.469516 30466 raft_consensus.cc:2749] T c7e3a6fa7dc74de286177a285dd3d877 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.470012 28733 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.470193 28733 ts_tablet_manager.cc:1403] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.061s user 0.003s sys 0.007s
I20260320 15:27:45.470465 28239 leader_election.cc:304] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:45.470793 30466 raft_consensus.cc:2749] T 8c4380f86ceb48b8b93eb2a5b9b302c7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.470885 28733 raft_consensus.cc:359] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.471069 28733 raft_consensus.cc:385] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.471143 28733 raft_consensus.cc:740] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.471374 28733 consensus_queue.cc:260] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.471910 28733 ts_tablet_manager.cc:1434] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.472214 28733 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.472416 30466 consensus_queue.cc:1048] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.473145 30466 consensus_queue.cc:1048] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.473888 28733 tablet_bootstrap.cc:654] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.462908 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 214) took 6066 ms. Trace:
I20260320 15:27:45.479697 29678 rpcz_store.cc:276] 0320 15:27:39.396208 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.396319 (+ 111us) service_pool.cc:225] Handling call
0320 15:27:45.462809 (+6066490us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:45.462821 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.462891 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5988449}
I20260320 15:27:45.480150 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 215) took 6083 ms. Trace:
I20260320 15:27:45.480224 29678 rpcz_store.cc:276] 0320 15:27:39.396556 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.396649 (+ 93us) service_pool.cc:225] Handling call
0320 15:27:45.480027 (+6083378us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:45.480040 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.480130 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6005616}
I20260320 15:27:45.480486 29678 tablet_replica.cc:333] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.480677 29678 raft_consensus.cc:2243] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.480845 29678 raft_consensus.cc:2272] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.481410 29678 ts_tablet_manager.cc:1916] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.486181 28736 tablet_bootstrap.cc:492] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.486383 28736 ts_tablet_manager.cc:1403] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260320 15:27:45.486994 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:45.487197 28440 raft_consensus.cc:3060] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.487254 28736 raft_consensus.cc:359] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.487428 28736 raft_consensus.cc:385] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.487488 28736 raft_consensus.cc:740] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.487730 28736 consensus_queue.cc:260] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.488231 28736 ts_tablet_manager.cc:1434] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.488548 28736 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.490401 28736 tablet_bootstrap.cc:654] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.490584 28440 raft_consensus.cc:2468] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:45.490762 29640 ts_tablet_manager.cc:1929] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.490885 29640 log.cc:1199] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/07ff84dedc074dabb4e8e2d8460066e7
I20260320 15:27:45.491227 29640 ts_tablet_manager.cc:1950] T 07ff84dedc074dabb4e8e2d8460066e7 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.494526 29643 ts_tablet_manager.cc:1929] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.494675 29643 log.cc:1199] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/54266dd376164673bffe829dd515b83d
I20260320 15:27:45.495136 29643 ts_tablet_manager.cc:1950] T 54266dd376164673bffe829dd515b83d P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:45.496381 28239 leader_election.cc:343] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:45.498449 28239 leader_election.cc:304] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:45.499707 28239 raft_consensus.cc:2686] T 16e31e11048b494ebe1fd4d89488b0cf P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:45.507357 30394 consensus_queue.cc:1048] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.507863 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 162) took 11261 ms. Trace:
I20260320 15:27:45.508038 29640 rpcz_store.cc:276] 0320 15:27:34.246347 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.254411 (+ 8064us) service_pool.cc:225] Handling call
0320 15:27:45.464709 (+11210298us) ts_tablet_manager.cc:1151] Deleting tablet 07ff84dedc074dabb4e8e2d8460066e7
0320 15:27:45.464719 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.469162 (+ 4443us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.490623 (+ 21461us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.507811 (+ 17188us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4969,"tablet-delete.queue_time_us":11210163}
I20260320 15:27:45.508726 29640 tablet_replica.cc:333] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.508713 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 07ff84dedc074dabb4e8e2d8460066e7 (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]) successfully deleted
I20260320 15:27:45.509004 29640 raft_consensus.cc:2243] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.509346 29640 raft_consensus.cc:2272] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.509954 29640 ts_tablet_manager.cc:1916] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.511046 30555 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:45.511121 29736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53210 (request call id 0) took 10550 ms. Trace:
I20260320 15:27:45.511305 30555 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:45.511473 28524 leader_election.cc:304] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.511679 30492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1caebf640d4ec0bf9b28c4f496ca18. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:45.511905 30492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:45.511950 30394 raft_consensus.cc:2804] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.512028 30394 raft_consensus.cc:493] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.512146 30394 raft_consensus.cc:3060] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.512851 30533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:45.513376 30533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [sys.catalog]: This master's current role is: FOLLOWER
I20260320 15:27:45.514520 28736 tablet_bootstrap.cc:492] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.514684 28736 ts_tablet_manager.cc:1403] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260320 15:27:45.516459 30394 raft_consensus.cc:515] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.517092 30394 leader_election.cc:290] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.518738 29678 ts_tablet_manager.cc:1929] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.518896 29678 log.cc:1199] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ac4c7dc6859048ce956927201df44a86
I20260320 15:27:45.519282 29678 ts_tablet_manager.cc:1950] T ac4c7dc6859048ce956927201df44a86 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.519687 30555 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:45.524483 28736 raft_consensus.cc:359] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.524765 28736 raft_consensus.cc:385] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.524838 28736 raft_consensus.cc:740] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.530705 28524 leader_election.cc:304] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.525142 28736 consensus_queue.cc:260] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.532512 28736 ts_tablet_manager.cc:1434] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260320 15:27:45.532840 28736 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.534471 30466 raft_consensus.cc:493] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.534683 30466 raft_consensus.cc:515] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.535420 30466 leader_election.cc:290] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:45.520252 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 216) took 6123 ms. Trace:
I20260320 15:27:45.535748 29678 rpcz_store.cc:276] 0320 15:27:39.397342 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.397645 (+ 303us) service_pool.cc:225] Handling call
0320 15:27:45.480433 (+6082788us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:45.480444 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.502643 (+ 22199us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.518612 (+ 15969us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.520209 (+ 1597us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6005975}
I20260320 15:27:45.535979 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.536231 28440 raft_consensus.cc:2393] T 44327000792444b89265f4a106eb11d0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:45.536228 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 218) took 6138 ms. Trace:
I20260320 15:27:45.536301 29678 rpcz_store.cc:276] 0320 15:27:39.398140 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.398231 (+ 91us) service_pool.cc:225] Handling call
0320 15:27:45.536116 (+6137885us) ts_tablet_manager.cc:1151] Deleting tablet 22b7f757ebdf456eb81085e5be54da4b
0320 15:27:45.536126 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.536211 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6061609}
I20260320 15:27:45.536543 29678 tablet_replica.cc:333] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.536763 29678 raft_consensus.cc:2243] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.536939 29678 raft_consensus.cc:2272] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.537429 30155 raft_consensus.cc:2804] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:45.537580 29678 ts_tablet_manager.cc:1916] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.537599 30155 raft_consensus.cc:697] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:45.537871 30155 consensus_queue.cc:237] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.545033 28736 tablet_bootstrap.cc:654] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.550179 29640 ts_tablet_manager.cc:1929] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.551808 28734 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.551981 28734 ts_tablet_manager.cc:1403] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.109s user 0.000s sys 0.012s
I20260320 15:27:45.552681 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.552891 28734 raft_consensus.cc:359] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.553056 28734 raft_consensus.cc:385] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.553109 28734 raft_consensus.cc:740] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.553313 30597 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260320 15:27:45.553375 28734 consensus_queue.cc:260] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.553833 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 204) took 8557 ms. Trace:
I20260320 15:27:45.553887 28734 ts_tablet_manager.cc:1434] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:45.553938 29643 rpcz_store.cc:276] 0320 15:27:36.996476 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.997047 (+ 571us) service_pool.cc:225] Handling call
0320 15:27:45.451521 (+8454474us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:45.451562 (+ 41us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.483974 (+ 32412us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.494397 (+ 10423us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.508556 (+ 14159us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x6160000e2954. stack: 00007ff37d36a076 00007ff37d369e30 00007ff37c3243ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007ff38926966a 00007ff389227a14 00007ff38a5bcc16 00007ff38d17ee74 00007ff38d1b1b44 00007ff38d192560 00007ff38d1922ed 0000000000a8b58c 00007ff37d3aff5f 00007ff37d3b327b
0320 15:27:45.553809 (+ 45253us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":29701504,"tablet-delete.queue_time_us":8454344}
I20260320 15:27:45.554180 28734 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.554381 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 195) took 8558 ms. Trace:
I20260320 15:27:45.554478 29643 rpcz_store.cc:276] 0320 15:27:36.995585 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.997436 (+ 1851us) service_pool.cc:225] Handling call
0320 15:27:45.554265 (+8556829us) ts_tablet_manager.cc:1151] Deleting tablet b395977eb0444bcf8563cd8774fab40f
0320 15:27:45.554274 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.554364 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8556682}
I20260320 15:27:45.554813 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 199) took 8558 ms. Trace:
I20260320 15:27:45.554893 29643 rpcz_store.cc:276] 0320 15:27:36.995982 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.997601 (+ 1619us) service_pool.cc:225] Handling call
0320 15:27:45.554701 (+8557100us) ts_tablet_manager.cc:1151] Deleting tablet ae3422cf33324ab89a2b44e27cae0eb2
0320 15:27:45.554710 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.554798 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8556940}
I20260320 15:27:45.555243 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 203) took 8558 ms. Trace:
I20260320 15:27:45.555342 29643 rpcz_store.cc:276] 0320 15:27:36.996385 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.997800 (+ 1415us) service_pool.cc:225] Handling call
0320 15:27:45.555160 (+8557360us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:45.555168 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.555228 (+ 60us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8557131}
I20260320 15:27:45.555807 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 54266dd376164673bffe829dd515b83d (table default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]) successfully deleted
I20260320 15:27:45.556135 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 213) took 8515 ms. Trace:
W20260320 15:27:45.556178 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b395977eb0444bcf8563cd8774fab40f because the tablet was not found. No further retry: Not found: Tablet not found: b395977eb0444bcf8563cd8774fab40f
I20260320 15:27:45.556217 29643 rpcz_store.cc:276] 0320 15:27:37.040378 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.040442 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:45.555918 (+8515476us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:45.556049 (+ 131us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.556116 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8515113}
W20260320 15:27:45.556401 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae3422cf33324ab89a2b44e27cae0eb2 already in progress: creating tablet
W20260320 15:27:45.556658 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b34048a722c3485490637965c60cc5dc because the tablet was not found. No further retry: Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
W20260320 15:27:45.556856 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because the tablet was not found. No further retry: Not found: Tablet not found: c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:45.559401 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 215) took 8516 ms. Trace:
I20260320 15:27:45.559505 29643 rpcz_store.cc:276] 0320 15:27:37.043252 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.043321 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:45.556468 (+8513147us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:45.559293 (+ 2825us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.559377 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8512854}
I20260320 15:27:45.559911 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 220) took 8502 ms. Trace:
I20260320 15:27:45.559988 29643 rpcz_store.cc:276] 0320 15:27:37.057097 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.057144 (+ 47us) service_pool.cc:225] Handling call
0320 15:27:45.559785 (+8502641us) ts_tablet_manager.cc:1151] Deleting tablet 197f1935099140a98c5eb801cf44046a
0320 15:27:45.559794 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.559871 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8502403}
I20260320 15:27:45.560357 29643 tablet_replica.cc:333] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.560564 29643 raft_consensus.cc:2243] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.560774 29643 raft_consensus.cc:2272] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.561357 29643 ts_tablet_manager.cc:1916] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.564935 28734 tablet_bootstrap.cc:654] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:45.566213 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because the tablet was not found. No further retry: Not found: Tablet not found: 2165371f4b5448c7a2625926082ee4c6
W20260320 15:27:45.566754 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 197f1935099140a98c5eb801cf44046a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 197f1935099140a98c5eb801cf44046a already in progress: creating tablet
I20260320 15:27:45.568053 29672 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" instance_seqno: 1774020437138719) as {username='slave'} at 127.27.74.2:38781; Asking this server to re-register.
I20260320 15:27:45.568885 29640 log.cc:1199] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:45.569608 29640 ts_tablet_manager.cc:1950] T fa5618796f1a4ff3ab4c1209dad0aede P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.569842 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 609 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:45.559916 28733 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.570252 28733 ts_tablet_manager.cc:1403] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.098s user 0.001s sys 0.013s
I20260320 15:27:45.570745 30456 raft_consensus.cc:493] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.570912 30456 raft_consensus.cc:515] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.571496 30456 leader_election.cc:290] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.570472 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 157) took 11330 ms. Trace:
I20260320 15:27:45.572095 29640 rpcz_store.cc:276] 0320 15:27:34.240484 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.241787 (+ 1303us) service_pool.cc:225] Handling call
0320 15:27:45.508642 (+11266855us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:45.508655 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.547078 (+ 38423us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.550024 (+ 2946us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.570433 (+ 20409us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11231748}
I20260320 15:27:45.573021 28736 tablet_bootstrap.cc:492] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.572526 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.573570 28382 leader_election.cc:343] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.574448 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet fa5618796f1a4ff3ab4c1209dad0aede (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
I20260320 15:27:45.574955 29640 tablet_replica.cc:333] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.575739 29640 raft_consensus.cc:2243] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.576133 29640 raft_consensus.cc:2272] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.576396 28736 ts_tablet_manager.cc:1403] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.002s
I20260320 15:27:45.577085 29672 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3ce5383d28b347ab979765b880e5773c" instance_seqno: 1774020436584169) as {username='slave'} at 127.27.74.1:47861; Asking this server to re-register.
I20260320 15:27:45.577301 28736 raft_consensus.cc:359] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.577458 28736 raft_consensus.cc:385] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.577519 28736 raft_consensus.cc:740] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.577734 28736 consensus_queue.cc:260] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.578269 28736 ts_tablet_manager.cc:1434] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:45.578574 28736 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.575352 28733 raft_consensus.cc:359] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.580032 28733 raft_consensus.cc:385] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.580199 28733 raft_consensus.cc:740] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.580515 28733 consensus_queue.cc:260] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.583300 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.584537 28734 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.584717 28734 ts_tablet_manager.cc:1403] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.002s
I20260320 15:27:45.585744 28734 raft_consensus.cc:359] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.585891 29640 ts_tablet_manager.cc:1916] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.586007 28734 raft_consensus.cc:385] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.586082 28734 raft_consensus.cc:740] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.586288 30466 raft_consensus.cc:493] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.586381 28734 consensus_queue.cc:260] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.586447 30466 raft_consensus.cc:515] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.586654 28733 ts_tablet_manager.cc:1434] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260320 15:27:45.586997 28734 ts_tablet_manager.cc:1434] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.587007 28733 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.587324 28734 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.587373 30466 leader_election.cc:290] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:45.587831 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.589500 28733 tablet_bootstrap.cc:654] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.590798 29678 ts_tablet_manager.cc:1929] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.590946 29678 log.cc:1199] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:45.591472 29678 ts_tablet_manager.cc:1950] T 7a4b5a658cd84b04b50b54973c065889 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.596271 28351 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:45.600764 28736 tablet_bootstrap.cc:654] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.606324 28734 tablet_bootstrap.cc:654] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.607779 29640 ts_tablet_manager.cc:1929] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.608696 29640 log.cc:1199] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/f0a31d385b81456fb46ea52083874fcf
I20260320 15:27:45.609761 29640 ts_tablet_manager.cc:1950] T f0a31d385b81456fb46ea52083874fcf P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.610967 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 166) took 11215 ms. Trace:
I20260320 15:27:45.607781 28494 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:45.612922 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:45.570770 29643 ts_tablet_manager.cc:1929] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:45.613161 28441 raft_consensus.cc:3060] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.613338 29643 log.cc:1199] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:45.613835 29643 ts_tablet_manager.cc:1950] T 5a20d7f22cfa403184e73b24d565b60a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:45.615116 29640 rpcz_store.cc:276] 0320 15:27:34.395577 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.395816 (+ 239us) service_pool.cc:225] Handling call
0320 15:27:45.574875 (+11179059us) ts_tablet_manager.cc:1151] Deleting tablet f0a31d385b81456fb46ea52083874fcf
0320 15:27:45.574888 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.603790 (+ 28902us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.607625 (+ 3835us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.610930 (+ 3305us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11178797}
I20260320 15:27:45.615705 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet f0a31d385b81456fb46ea52083874fcf (table default.Table_af7bda6fe8e14815bfd5bf848ac65d4c [id=26031596e6e346278c132a80bc8f104b]) successfully deleted
I20260320 15:27:45.616382 28441 raft_consensus.cc:2468] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:45.616449 29640 tablet_replica.cc:333] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.616676 29640 raft_consensus.cc:2243] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.616981 29640 raft_consensus.cc:2272] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.620497 28733 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.620577 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 222) took 8545 ms. Trace:
I20260320 15:27:45.620673 28733 ts_tablet_manager.cc:1403] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.000s
I20260320 15:27:45.620702 29643 rpcz_store.cc:276] 0320 15:27:37.074969 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.075611 (+ 642us) service_pool.cc:225] Handling call
0320 15:27:45.560297 (+8484686us) ts_tablet_manager.cc:1151] Deleting tablet 5a20d7f22cfa403184e73b24d565b60a
0320 15:27:45.560306 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.567467 (+ 7161us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.570620 (+ 3153us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.620525 (+ 49905us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8484383}
I20260320 15:27:45.621183 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 227) took 7080 ms. Trace:
I20260320 15:27:45.621358 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]) successfully deleted
I20260320 15:27:45.621301 29643 rpcz_store.cc:276] 0320 15:27:38.540914 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.540976 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:45.621004 (+7080028us) ts_tablet_manager.cc:1151] Deleting tablet 20eec704800e4d8b924d97556cc2700e
0320 15:27:45.621014 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.621127 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7079772}
I20260320 15:27:45.621694 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 230) took 6495 ms. Trace:
I20260320 15:27:45.621809 29643 rpcz_store.cc:276] 0320 15:27:39.126080 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.126557 (+ 477us) service_pool.cc:225] Handling call
0320 15:27:45.621578 (+6495021us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:45.621588 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.621650 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6494732}
I20260320 15:27:45.622237 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 231) took 6496 ms. Trace:
I20260320 15:27:45.622334 29643 rpcz_store.cc:276] 0320 15:27:39.126205 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.127523 (+ 1318us) service_pool.cc:225] Handling call
0320 15:27:45.622132 (+6494609us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:45.622141 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.622193 (+ 52us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6494343}
I20260320 15:27:45.622771 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 232) took 6496 ms. Trace:
W20260320 15:27:45.622746 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 20eec704800e4d8b924d97556cc2700e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20eec704800e4d8b924d97556cc2700e already in progress: creating tablet
I20260320 15:27:45.622862 29643 rpcz_store.cc:276] 0320 15:27:39.126310 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.127834 (+ 1524us) service_pool.cc:225] Handling call
0320 15:27:45.622669 (+6494835us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:45.622677 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.622728 (+ 51us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6494681}
W20260320 15:27:45.623190 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because the tablet was not found. No further retry: Not found: Tablet not found: b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:45.623327 29640 ts_tablet_manager.cc:1916] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.623389 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 233) took 6496 ms. Trace:
W20260320 15:27:45.623479 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f70530cd63a0420fb51527759c951e31 because the tablet was not found. No further retry: Not found: Tablet not found: f70530cd63a0420fb51527759c951e31
I20260320 15:27:45.623478 29643 rpcz_store.cc:276] 0320 15:27:39.126404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.128020 (+ 1616us) service_pool.cc:225] Handling call
0320 15:27:45.623294 (+6495274us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:45.623303 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.623356 (+ 53us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6495153}
W20260320 15:27:45.623780 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because the tablet was not found. No further retry: Not found: Tablet not found: c323b7bcb671464ea6ff584a633f10c0
W20260320 15:27:45.624003 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 4becc525b59d421caf79d8cd42052118 because the tablet was not found. No further retry: Not found: Tablet not found: 4becc525b59d421caf79d8cd42052118
I20260320 15:27:45.624125 28733 raft_consensus.cc:359] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.624341 28733 raft_consensus.cc:385] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.624414 28733 raft_consensus.cc:740] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.624680 28733 consensus_queue.cc:260] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.625417 28733 ts_tablet_manager.cc:1434] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260320 15:27:45.625823 28733 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.628573 28733 tablet_bootstrap.cc:654] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.636729 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e9791ac61b34fc09a8292c79f0c4032" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.637459 29640 ts_tablet_manager.cc:1929] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.637609 29640 log.cc:1199] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/8d92e17cea044541968356ae993be1e7
I20260320 15:27:45.637979 29640 ts_tablet_manager.cc:1950] T 8d92e17cea044541968356ae993be1e7 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.639921 30466 raft_consensus.cc:493] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.640075 30466 raft_consensus.cc:515] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:45.640698 28733 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.640730 30466 leader_election.cc:290] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:45.640867 28733 ts_tablet_manager.cc:1403] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260320 15:27:45.641932 28733 raft_consensus.cc:359] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.642189 28733 raft_consensus.cc:385] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.642298 28733 raft_consensus.cc:740] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.642827 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.642426 30394 consensus_queue.cc:1048] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260320 15:27:45.642592 28733 consensus_queue.cc:260] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.643306 28441 raft_consensus.cc:2393] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:45.643841 28733 ts_tablet_manager.cc:1434] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260320 15:27:45.647053 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 217) took 6247 ms. Trace:
I20260320 15:27:45.647280 29678 rpcz_store.cc:276] 0320 15:27:39.397854 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.398944 (+ 1090us) service_pool.cc:225] Handling call
0320 15:27:45.536501 (+6137557us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:45.536509 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.579720 (+ 43211us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.590670 (+ 10950us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.644909 (+ 54239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6061943}
I20260320 15:27:45.650434 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 219) took 5958 ms. Trace:
I20260320 15:27:45.650624 29678 rpcz_store.cc:276] 0320 15:27:39.692311 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.692429 (+ 118us) service_pool.cc:225] Handling call
0320 15:27:45.650303 (+5957874us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:45.650318 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.650412 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5957641}
I20260320 15:27:45.651044 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 220) took 5958 ms. Trace:
I20260320 15:27:45.651185 29678 rpcz_store.cc:276] 0320 15:27:39.692744 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.692844 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:45.650953 (+5958109us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:45.650964 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.651027 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5957961}
I20260320 15:27:45.651574 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 222) took 5954 ms. Trace:
I20260320 15:27:45.651937 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 234) took 6525 ms. Trace:
I20260320 15:27:45.652077 29678 rpcz_store.cc:276] 0320 15:27:39.696998 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.697100 (+ 102us) service_pool.cc:225] Handling call
0320 15:27:45.651477 (+5954377us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:45.651484 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.651560 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5954152}
I20260320 15:27:45.652458 29678 tablet_replica.cc:333] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.652606 29643 rpcz_store.cc:276] 0320 15:27:39.126487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.128167 (+ 1680us) service_pool.cc:225] Handling call
0320 15:27:45.651808 (+6523641us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:45.651819 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.651891 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6523520}
I20260320 15:27:45.653016 29678 raft_consensus.cc:2243] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.653470 29678 raft_consensus.cc:2272] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.653828 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 235) took 6527 ms. Trace:
I20260320 15:27:45.654672 29678 ts_tablet_manager.cc:1916] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.654901 28734 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.655234 28734 ts_tablet_manager.cc:1403] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
I20260320 15:27:45.656165 28734 raft_consensus.cc:359] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.656448 28734 raft_consensus.cc:385] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.656584 28734 raft_consensus.cc:740] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.656883 28734 consensus_queue.cc:260] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.657450 28734 ts_tablet_manager.cc:1434] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:45.657958 28734 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.660197 29672 ts_manager.cc:194] Registered new tserver with Master: d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:45.654831 29643 rpcz_store.cc:276] 0320 15:27:39.126575 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.128311 (+ 1736us) service_pool.cc:225] Handling call
0320 15:27:45.653705 (+6525394us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:45.653716 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.653795 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6525245}
W20260320 15:27:45.661585 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 16e31e11048b494ebe1fd4d89488b0cf because the tablet was not found. No further retry: Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:45.661672 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 236) took 6535 ms. Trace:
I20260320 15:27:45.661779 29643 rpcz_store.cc:276] 0320 15:27:39.126666 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.128490 (+ 1824us) service_pool.cc:225] Handling call
0320 15:27:45.661536 (+6533046us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:45.661549 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.661634 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6532881}
W20260320 15:27:45.662213 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ac4c7dc6859048ce956927201df44a86 because the tablet was not found. No further retry: Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:45.663319 28524 leader_election.cc:304] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.663720 28734 tablet_bootstrap.cc:654] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.664270 30395 raft_consensus.cc:2804] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:45.664413 30395 raft_consensus.cc:697] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:45.664651 30395 consensus_queue.cc:237] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.664919 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 237) took 6538 ms. Trace:
I20260320 15:27:45.665024 29643 rpcz_store.cc:276] 0320 15:27:39.126798 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.128681 (+ 1883us) service_pool.cc:225] Handling call
0320 15:27:45.663935 (+6535254us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:45.664778 (+ 843us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.664863 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6535097}
I20260320 15:27:45.667169 29643 tablet_replica.cc:333] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.667378 29643 raft_consensus.cc:2243] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.667568 29643 raft_consensus.cc:2272] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.668156 29643 ts_tablet_manager.cc:1916] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.669988 29678 ts_tablet_manager.cc:1929] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.670094 29678 log.cc:1199] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/71ec63fb080545de838803033f204fab
I20260320 15:27:45.670413 29678 ts_tablet_manager.cc:1950] T 71ec63fb080545de838803033f204fab P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.671195 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 223) took 5968 ms. Trace:
I20260320 15:27:45.671276 29678 rpcz_store.cc:276] 0320 15:27:39.703513 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.704256 (+ 743us) service_pool.cc:225] Handling call
0320 15:27:45.652407 (+5948151us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:45.652417 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.666858 (+ 14441us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.669879 (+ 3021us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.671171 (+ 1292us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5947934}
I20260320 15:27:45.674477 29643 ts_tablet_manager.cc:1929] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:45.674631 29643 log.cc:1199] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:45.678541 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 224) took 5974 ms. Trace:
W20260320 15:27:45.680518 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because the tablet was not found. No further retry: Not found: Tablet not found: bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:45.680765 29678 rpcz_store.cc:276] 0320 15:27:39.704011 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.704510 (+ 499us) service_pool.cc:225] Handling call
0320 15:27:45.678347 (+5973837us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:45.678370 (+ 23us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.678507 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5966904}
I20260320 15:27:45.681282 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 225) took 5977 ms. Trace:
I20260320 15:27:45.681448 29678 rpcz_store.cc:276] 0320 15:27:39.704326 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.704700 (+ 374us) service_pool.cc:225] Handling call
0320 15:27:45.681176 (+5976476us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:45.681189 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.681264 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5976364}
I20260320 15:27:45.681634 30394 consensus_queue.cc:1048] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.682040 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 226) took 5977 ms. Trace:
I20260320 15:27:45.683022 29678 rpcz_store.cc:276] 0320 15:27:39.704599 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.705278 (+ 679us) service_pool.cc:225] Handling call
0320 15:27:45.681946 (+5976668us) ts_tablet_manager.cc:1151] Deleting tablet 591ccc1d349e4fd599d5e343f91defe2
0320 15:27:45.681958 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.682024 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5976515}
I20260320 15:27:45.685685 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 228) took 5980 ms. Trace:
I20260320 15:27:45.659307 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.687747 29678 rpcz_store.cc:276] 0320 15:27:39.705322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.705528 (+ 206us) service_pool.cc:225] Handling call
0320 15:27:45.685562 (+5980034us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:45.685574 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.685668 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5979900}
I20260320 15:27:45.689407 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 229) took 5836 ms. Trace:
I20260320 15:27:45.689675 29678 rpcz_store.cc:276] 0320 15:27:39.852820 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.860240 (+ 7420us) service_pool.cc:225] Handling call
0320 15:27:45.689138 (+5828898us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:45.689156 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.689382 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5811696}
I20260320 15:27:45.690373 29678 tablet_replica.cc:333] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.690411 28734 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.690871 28734 ts_tablet_manager.cc:1403] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.007s
I20260320 15:27:45.691057 29678 raft_consensus.cc:2243] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.691946 28734 raft_consensus.cc:359] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.693248 28734 raft_consensus.cc:385] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.693518 28734 raft_consensus.cc:740] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.692516 29643 ts_tablet_manager.cc:1950] T cea3bf14653f4555acfed1577a79ee4f P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:45.694729 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 241) took 6551 ms. Trace:
I20260320 15:27:45.695436 29643 rpcz_store.cc:276] 0320 15:27:39.142828 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.143004 (+ 176us) service_pool.cc:225] Handling call
0320 15:27:45.665399 (+6522395us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:45.667094 (+ 1695us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.671317 (+ 4223us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.674344 (+ 3027us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.694680 (+ 20336us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6522101}
I20260320 15:27:45.692016 29678 raft_consensus.cc:2272] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.696142 30456 raft_consensus.cc:493] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.695101 28734 consensus_queue.cc:260] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.696326 30456 raft_consensus.cc:515] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.697000 30456 leader_election.cc:290] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:45.699083 28523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260320 15:27:45.699612 30466 raft_consensus.cc:670] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260320 15:27:45.699762 28734 ts_tablet_manager.cc:1434] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260320 15:27:45.699939 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet cea3bf14653f4555acfed1577a79ee4f (table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]) successfully deleted
I20260320 15:27:45.700105 28734 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.660050 29681 ts_manager.cc:194] Registered new tserver with Master: 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.702023 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 240) took 6559 ms. Trace:
I20260320 15:27:45.702818 29643 rpcz_store.cc:276] 0320 15:27:39.142693 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.143345 (+ 652us) service_pool.cc:225] Handling call
0320 15:27:45.701870 (+6558525us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:45.701882 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.701981 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6558327}
W20260320 15:27:45.703372 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because the tablet was not found. No further retry: Not found: Tablet not found: 9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:45.703878 29678 ts_tablet_manager.cc:1916] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.705372 28736 tablet_bootstrap.cc:492] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.705556 28736 ts_tablet_manager.cc:1403] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.127s user 0.007s sys 0.028s
I20260320 15:27:45.706058 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:45.706396 28736 raft_consensus.cc:359] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.706609 28736 raft_consensus.cc:385] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.706677 28736 raft_consensus.cc:740] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.706900 28736 consensus_queue.cc:260] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.707679 28736 ts_tablet_manager.cc:1434] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:45.708036 28736 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.710265 28736 tablet_bootstrap.cc:654] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.711225 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.711954 28382 leader_election.cc:343] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.713263 28733 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.715263 28733 tablet_bootstrap.cc:654] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.716140 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c453eaf49c4171b0f93d3b26690d83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.716876 29678 ts_tablet_manager.cc:1929] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:45.716984 29678 log.cc:1199] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:45.717316 29678 ts_tablet_manager.cc:1950] T cea3bf14653f4555acfed1577a79ee4f P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.718137 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 230) took 5857 ms. Trace:
I20260320 15:27:45.718230 29678 rpcz_store.cc:276] 0320 15:27:39.860616 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.861014 (+ 398us) service_pool.cc:225] Handling call
0320 15:27:45.690278 (+5829264us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:45.690292 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.713144 (+ 22852us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.716756 (+ 3612us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.718115 (+ 1359us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5812755}
I20260320 15:27:45.718632 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 231) took 5816 ms. Trace:
I20260320 15:27:45.718705 29678 rpcz_store.cc:276] 0320 15:27:39.902471 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.902529 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:45.718516 (+5815987us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:45.718525 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.718615 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5815806}
I20260320 15:27:45.719000 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 232) took 5816 ms. Trace:
I20260320 15:27:45.721982 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 170) took 10131 ms. Trace:
I20260320 15:27:45.722133 29640 rpcz_store.cc:276] 0320 15:27:35.590137 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.590203 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:45.616363 (+10026160us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e17cea044541968356ae993be1e7
0320 15:27:45.616375 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.629536 (+ 13161us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.637308 (+ 7772us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.721933 (+ 84625us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10025946}
I20260320 15:27:45.722563 29640 tablet_replica.cc:333] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.722788 29640 raft_consensus.cc:2243] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.722813 28902 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 138) took 1565 ms. Trace:
I20260320 15:27:45.722998 29640 raft_consensus.cc:2272] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.722950 28902 rpcz_store.cc:276] 0320 15:27:44.156960 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.324741 (+1167781us) service_pool.cc:225] Handling call
0320 15:27:45.340605 (+ 15864us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.722721 (+382116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35703552}
I20260320 15:27:45.723280 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 4190329df1b644c392ff8cbace9aae7a
I20260320 15:27:45.724438 30454 raft_consensus.cc:493] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.724625 30454 raft_consensus.cc:515] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.725317 30454 leader_election.cc:290] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.727981 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.728459 28906 catalog_manager.cc:6741] ReplaceTablet: tablet 4190329df1b644c392ff8cbace9aae7a deleted and replaced by tablet 806d635bb5594f0bb826b9b31468b485
I20260320 15:27:45.728631 28906 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 132) took 2643 ms. Trace:
I20260320 15:27:45.728760 28906 rpcz_store.cc:276] 0320 15:27:43.085065 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.231234 (+2146169us) service_pool.cc:225] Handling call
0320 15:27:45.269167 (+ 37933us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.728588 (+459421us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.714568 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.714699 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.714707 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:45.714709 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.714826 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.714841 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.714846 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.714929 (+ 83us) tablet.cc:700] Row locks acquired
0320 15:27:45.714931 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:45.714972 (+ 41us) write_op.cc:270] Start()
0320 15:27:45.715012 (+ 40us) write_op.cc:276] Timestamp: P: 1774020465714914 usec, L: 0
0320 15:27:45.715017 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.715319 (+ 302us) log.cc:844] Serialized 622 byte log entry
0320 15:27:45.721717 (+ 6398us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.721852 (+ 135us) write_op.cc:301] APPLY: starting
0320 15:27:45.721874 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.721928 (+ 54us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.721958 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.722098 (+ 140us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.722168 (+ 70us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.722176 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:45.722281 (+ 105us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.722360 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.722386 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.722394 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.722480 (+ 86us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.722484 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4547,"spinlock_wait_cycles":86139648,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":616,"apply.run_wall_time_us":615,"num_ops":2,"prepare.queue_time_us":71,"prepare.run_cpu_time_us":874,"prepare.run_wall_time_us":874,"raft.queue_time_us":1931,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":331,"replication_time_us":6660}]]}
I20260320 15:27:45.729650 28906 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_799abb7057d3405db01b42c86431f01d" } modify_external_catalogs: true
I20260320 15:27:45.729871 28906 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_799abb7057d3405db01b42c86431f01d" } modify_external_catalogs: true
W20260320 15:27:45.730269 28382 leader_election.cc:343] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.732702 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]) successfully deleted
I20260320 15:27:45.732908 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 609 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:45.733968 29657 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892; no voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:45.735009 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 4190329df1b644c392ff8cbace9aae7a with delete_type TABLET_DATA_DELETED (replaced by tablet 806d635bb5594f0bb826b9b31468b485) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.735107 30555 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260320 15:27:45.737214 30454 raft_consensus.cc:493] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.737386 30454 raft_consensus.cc:515] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.737998 30454 leader_election.cc:290] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.738903 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.739387 28382 leader_election.cc:343] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.740502 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]) successfully deleted
W20260320 15:27:45.740799 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because the tablet was not found. No further retry: Not found: Tablet not found: c752fb99ef494bda9d144d0e180e3100
W20260320 15:27:45.741025 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 197f1935099140a98c5eb801cf44046a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 197f1935099140a98c5eb801cf44046a already in progress: creating tablet
W20260320 15:27:45.742238 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because the tablet was not found. No further retry: Not found: Tablet not found: b82c47ae3786401d931bb0b486ca3570
W20260320 15:27:45.742332 28524 leader_election.cc:343] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:45.742520 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 20eec704800e4d8b924d97556cc2700e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 20eec704800e4d8b924d97556cc2700e already in progress: creating tablet
I20260320 15:27:45.742905 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet fa5618796f1a4ff3ab4c1209dad0aede (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]) successfully deleted
W20260320 15:27:45.743181 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b03e7555b9a646b599acb0c7dff90c9a because the tablet was not found. No further retry: Not found: Tablet not found: b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:45.743741 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 4190329df1b644c392ff8cbace9aae7a with delete_type TABLET_DATA_DELETED (replaced by tablet 806d635bb5594f0bb826b9b31468b485) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:45.743821 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet f70530cd63a0420fb51527759c951e31 because the tablet was not found. No further retry: Not found: Tablet not found: f70530cd63a0420fb51527759c951e31
W20260320 15:27:45.744000 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because the tablet was not found. No further retry: Not found: Tablet not found: bb7a910bd2904a40ae63608377206c7e
W20260320 15:27:45.744194 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fa22d207ad8a4555aa9573b15fa1fe1a because the tablet was not found. No further retry: Not found: Tablet not found: fa22d207ad8a4555aa9573b15fa1fe1a
W20260320 15:27:45.744377 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bcec6175c1ca49f5b682a29d5c1e235a because the tablet was not found. No further retry: Not found: Tablet not found: bcec6175c1ca49f5b682a29d5c1e235a
W20260320 15:27:45.744599 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e2586d40343b4365ab8a67cce2e1c6ca because the tablet was not found. No further retry: Not found: Tablet not found: e2586d40343b4365ab8a67cce2e1c6ca
I20260320 15:27:45.744757 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 16e31e11048b494ebe1fd4d89488b0cf (table default.Table_94aca51eb5d64663a167b3806116218e [id=27ad5135c9c54016ba6be2435a356efa]) successfully deleted
W20260320 15:27:45.744949 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because the tablet was not found. No further retry: Not found: Tablet not found: c323b7bcb671464ea6ff584a633f10c0
W20260320 15:27:45.744782 28524 leader_election.cc:343] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.746758 29678 rpcz_store.cc:276] 0320 15:27:39.902932 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.902975 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:45.718899 (+5815924us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:45.718907 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.718987 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5815790}
I20260320 15:27:45.747270 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 233) took 5844 ms. Trace:
I20260320 15:27:45.747373 29678 rpcz_store.cc:276] 0320 15:27:39.903277 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.903319 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:45.747086 (+5843767us) ts_tablet_manager.cc:1151] Deleting tablet 48c1002cbdd845e2b96dcb57c450cc6d
0320 15:27:45.747099 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.747238 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5843629}
I20260320 15:27:45.747730 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 236) took 5436 ms. Trace:
I20260320 15:27:45.747808 29678 rpcz_store.cc:276] 0320 15:27:40.311574 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.311646 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:45.747619 (+5435973us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:45.747628 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.747714 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5435800}
I20260320 15:27:45.748092 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 234) took 5437 ms. Trace:
I20260320 15:27:45.748170 29678 rpcz_store.cc:276] 0320 15:27:40.310540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.310592 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:45.748008 (+5437416us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:45.748016 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.748080 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5436111}
I20260320 15:27:45.748440 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 235) took 5437 ms. Trace:
I20260320 15:27:45.748512 29678 rpcz_store.cc:276] 0320 15:27:40.311259 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.311306 (+ 47us) service_pool.cc:225] Handling call
0320 15:27:45.748369 (+5437063us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:45.748377 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.748427 (+ 50us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5436431}
I20260320 15:27:45.748785 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 237) took 5436 ms. Trace:
I20260320 15:27:45.748863 29678 rpcz_store.cc:276] 0320 15:27:40.312067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.312109 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:45.748699 (+5436590us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:45.748706 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.748771 (+ 65us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5436411}
I20260320 15:27:45.749152 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 238) took 5436 ms. Trace:
I20260320 15:27:45.749233 29678 rpcz_store.cc:276] 0320 15:27:40.312449 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.312517 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:45.749081 (+5436564us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:45.749089 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.749137 (+ 48us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5436358}
I20260320 15:27:45.749533 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 239) took 5435 ms. Trace:
I20260320 15:27:45.749612 29678 rpcz_store.cc:276] 0320 15:27:40.313763 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.313964 (+ 201us) service_pool.cc:225] Handling call
0320 15:27:45.749442 (+5435478us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:45.749451 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.749518 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5435292}
I20260320 15:27:45.749857 29678 tablet_replica.cc:333] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.750051 29678 raft_consensus.cc:2243] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:45.748791 28524 leader_election.cc:343] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.750218 29678 raft_consensus.cc:2272] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.754657 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44327000792444b89265f4a106eb11d0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.755285 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet d04ab44039124f36ac60601098586b4e
I20260320 15:27:45.755775 28892 catalog_manager.cc:6741] ReplaceTablet: tablet d04ab44039124f36ac60601098586b4e deleted and replaced by tablet f77a32615c954da581e9a9e4cb45c497
I20260320 15:27:45.755889 28892 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 140) took 1165 ms. Trace:
I20260320 15:27:45.755993 28892 rpcz_store.cc:276] 0320 15:27:44.590248 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.373864 (+783616us) service_pool.cc:225] Handling call
0320 15:27:45.755860 (+381996us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.737657 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.737793 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.737801 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:45.737805 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:45.737918 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.737931 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.737935 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.738024 (+ 89us) tablet.cc:700] Row locks acquired
0320 15:27:45.738027 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.738066 (+ 39us) write_op.cc:270] Start()
0320 15:27:45.738103 (+ 37us) write_op.cc:276] Timestamp: P: 1774020465738007 usec, L: 0
0320 15:27:45.738106 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.738301 (+ 195us) log.cc:844] Serialized 622 byte log entry
0320 15:27:45.753703 (+ 15402us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.753842 (+ 139us) write_op.cc:301] APPLY: starting
0320 15:27:45.753859 (+ 17us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.753910 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.753941 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.754046 (+ 105us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.754111 (+ 65us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.754118 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.754196 (+ 78us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.754260 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.754281 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:45.754287 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.754359 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.754365 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":504,"apply.run_wall_time_us":504,"num_ops":2,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":720,"prepare.run_wall_time_us":721,"replication_time_us":15563}]]}
I20260320 15:27:45.756737 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_ed5104d8fb4f435e87290e89a34f96e1" } modify_external_catalogs: true
I20260320 15:27:45.756942 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_ed5104d8fb4f435e87290e89a34f96e1" } modify_external_catalogs: true
I20260320 15:27:45.757017 29678 ts_tablet_manager.cc:1916] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.758814 30454 consensus_queue.cc:1048] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:45.760125 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet d04ab44039124f36ac60601098586b4e with delete_type TABLET_DATA_DELETED (replaced by tablet f77a32615c954da581e9a9e4cb45c497) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:45.769229 29678 ts_tablet_manager.cc:1929] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.769529 29678 log.cc:1199] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6292904782fd4b619483135e58b66684
I20260320 15:27:45.770051 29678 ts_tablet_manager.cc:1950] T 6292904782fd4b619483135e58b66684 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:45.770501 28383 leader_election.cc:343] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.770695 28383 leader_election.cc:304] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:45.771062 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 4becc525b59d421caf79d8cd42052118 because the tablet was not found. No further retry: Not found: Tablet not found: 4becc525b59d421caf79d8cd42052118
I20260320 15:27:45.771092 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 240) took 5457 ms. Trace:
I20260320 15:27:45.771268 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet d04ab44039124f36ac60601098586b4e with delete_type TABLET_DATA_DELETED (replaced by tablet f77a32615c954da581e9a9e4cb45c497) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:45.771194 29678 rpcz_store.cc:276] 0320 15:27:40.313921 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.314233 (+ 312us) service_pool.cc:225] Handling call
0320 15:27:45.749808 (+5435575us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:45.749816 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.760193 (+ 10377us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.769070 (+ 8877us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.771067 (+ 1997us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5435401}
I20260320 15:27:45.771450 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ac4c7dc6859048ce956927201df44a86 (table default.Table_3c78548647f4450687bd63dcc8123436 [id=5ae5774e3c9f4ee088656c0ad7b46e3e]) successfully deleted
I20260320 15:27:45.771656 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 241) took 5443 ms. Trace:
I20260320 15:27:45.771802 30454 raft_consensus.cc:2749] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.771754 29678 rpcz_store.cc:276] 0320 15:27:40.328209 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.328282 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:45.771478 (+5443196us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:45.771490 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.771631 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5442843}
I20260320 15:27:45.772148 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 242) took 5443 ms. Trace:
I20260320 15:27:45.772250 29678 rpcz_store.cc:276] 0320 15:27:40.328786 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.328831 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:45.772049 (+5443218us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:45.772058 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.772134 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5442991}
I20260320 15:27:45.772548 29678 tablet_replica.cc:333] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
W20260320 15:27:45.772660 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 22b7f757ebdf456eb81085e5be54da4b because the tablet was not found. No further retry: Not found: Tablet not found: 22b7f757ebdf456eb81085e5be54da4b
I20260320 15:27:45.772764 29678 raft_consensus.cc:2243] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.775138 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 242) took 6627 ms. Trace:
I20260320 15:27:45.775661 29643 rpcz_store.cc:276] 0320 15:27:39.148097 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.148173 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:45.703270 (+6555097us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:45.774963 (+ 71693us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.775083 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6554768}
W20260320 15:27:45.776511 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ea95b7a1b30e42cebcbfcd551c58f679 because the tablet was not found. No further retry: Not found: Tablet not found: ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:45.777884 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 245) took 6536 ms. Trace:
I20260320 15:27:45.778760 29678 raft_consensus.cc:2272] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.779111 30454 raft_consensus.cc:493] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.779332 30454 raft_consensus.cc:515] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.780006 29678 ts_tablet_manager.cc:1916] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.780509 30454 leader_election.cc:290] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.780814 28734 tablet_bootstrap.cc:654] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.787345 29640 ts_tablet_manager.cc:1916] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.789736 30355 raft_consensus.cc:493] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.790023 30355 raft_consensus.cc:515] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.790859 30355 leader_election.cc:290] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.791772 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:45.793128 28382 leader_election.cc:343] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.794935 29640 ts_tablet_manager.cc:1929] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:45.795115 29640 log.cc:1199] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/27ccc693e7704650bd77e0c4ca11862e
W20260320 15:27:45.801406 28383 leader_election.cc:343] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.801610 28383 leader_election.cc:304] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:45.802471 30454 raft_consensus.cc:2749] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.804327 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.804625 28441 raft_consensus.cc:2468] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:45.807125 28383 leader_election.cc:343] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.807332 28383 leader_election.cc:304] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:45.807878 30454 raft_consensus.cc:2749] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:45.779345 29643 rpcz_store.cc:276] 0320 15:27:39.240321 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.240383 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:45.776918 (+6536535us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:45.776932 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.777010 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6524601}
W20260320 15:27:45.809921 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because the tablet was not found. No further retry: Not found: Tablet not found: 66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:45.810274 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 7a4b5a658cd84b04b50b54973c065889 (table default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]) successfully deleted
I20260320 15:27:45.810511 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 253) took 5987 ms. Trace:
W20260320 15:27:45.810573 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because the tablet was not found. No further retry: Not found: Tablet not found: f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:45.810660 29643 rpcz_store.cc:276] 0320 15:27:39.822806 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.822898 (+ 92us) service_pool.cc:225] Handling call
0320 15:27:45.810320 (+5987422us) ts_tablet_manager.cc:1151] Deleting tablet 48c1002cbdd845e2b96dcb57c450cc6d
0320 15:27:45.810336 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.810463 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5987130}
W20260320 15:27:45.810994 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet a4bfb87d5d5741adb0ad2f2a88f4867f because the tablet was not found. No further retry: Not found: Tablet not found: a4bfb87d5d5741adb0ad2f2a88f4867f
W20260320 15:27:45.811239 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because the tablet was not found. No further retry: Not found: Tablet not found: 7394e941c99e4358ad7a33db84b15348
W20260320 15:27:45.811592 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 48c1002cbdd845e2b96dcb57c450cc6d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48c1002cbdd845e2b96dcb57c450cc6d already in progress: creating tablet
I20260320 15:27:45.813829 28736 tablet_bootstrap.cc:492] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:45.814010 28736 ts_tablet_manager.cc:1403] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.106s user 0.003s sys 0.002s
I20260320 15:27:45.814230 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 71ec63fb080545de838803033f204fab (table default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]) successfully deleted
W20260320 15:27:45.814343 28383 leader_election.cc:343] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:45.814881 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because the tablet was not found. No further retry: Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:45.814836 28736 raft_consensus.cc:359] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.815029 28736 raft_consensus.cc:385] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.815114 28736 raft_consensus.cc:740] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:45.815316 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 591ccc1d349e4fd599d5e343f91defe2 because the tablet was not found. No further retry: Not found: Tablet not found: 591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:45.815333 28736 consensus_queue.cc:260] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.815894 28736 ts_tablet_manager.cc:1434] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:45.819624 30466 raft_consensus.cc:493] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.819780 30466 raft_consensus.cc:515] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.820418 30466 leader_election.cc:290] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:45.823029 29678 ts_tablet_manager.cc:1929] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260320 15:27:45.823168 29678 log.cc:1199] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:45.823570 29678 ts_tablet_manager.cc:1950] T 9dbd3b0e92a744dcbc132037c67303ab P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.827781 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.827921 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:45.828135 28441 raft_consensus.cc:2468] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:45.828642 28240 leader_election.cc:304] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.828930 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 256) took 5447 ms. Trace:
I20260320 15:27:45.829012 30466 raft_consensus.cc:2804] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.829078 30466 raft_consensus.cc:493] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.829133 30466 raft_consensus.cc:3060] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.829053 29643 rpcz_store.cc:276] 0320 15:27:40.381899 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.382051 (+ 152us) service_pool.cc:225] Handling call
0320 15:27:45.828802 (+5446751us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:45.828813 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.828904 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5446562}
I20260320 15:27:45.832019 30466 raft_consensus.cc:515] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.832407 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 258) took 5449 ms. Trace:
I20260320 15:27:45.832518 29643 rpcz_store.cc:276] 0320 15:27:40.382899 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.382951 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:45.832284 (+5449333us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:45.832298 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.832383 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5449131}
I20260320 15:27:45.833201 30466 leader_election.cc:290] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:45.833532 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:45.833755 28441 raft_consensus.cc:3060] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.836601 28441 raft_consensus.cc:2468] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:45.837149 28240 leader_election.cc:304] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.837477 30454 raft_consensus.cc:493] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.837637 30454 raft_consensus.cc:515] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.838579 30454 leader_election.cc:290] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.839160 30454 raft_consensus.cc:493] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.839313 30454 raft_consensus.cc:515] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.839448 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 6725124b62ad4171a2cfe072687b8c3d (DEFAULT_TABLE table=default.Table_a3cd321472b34384aadb6e966f5402da [id=65f2ef6dd1884f6e974f3b1c77e1b567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.840008 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6725124b62ad4171a2cfe072687b8c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.840037 30454 leader_election.cc:290] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.849551 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 52a9b7b73bc04a7bb937d863c375b10f (DEFAULT_TABLE table=default.Table_5228be23ce9740aa912d56fce56ec7ed [id=e4a61dea95f14ac09c99dfd92c90ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.850229 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a9b7b73bc04a7bb937d863c375b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.853991 28736 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:45.854180 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:45.855070 30466 raft_consensus.cc:2804] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:45.855561 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 243) took 5522 ms. Trace:
I20260320 15:27:45.855680 29678 rpcz_store.cc:276] 0320 15:27:40.333077 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.333151 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:45.772501 (+5439350us) ts_tablet_manager.cc:1151] Deleting tablet 9dbd3b0e92a744dcbc132037c67303ab
0320 15:27:45.772509 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.812693 (+ 40184us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.822900 (+ 10207us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.855516 (+ 32616us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1089,"tablet-delete.queue_time_us":5439083}
I20260320 15:27:45.856841 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 248) took 5335 ms. Trace:
I20260320 15:27:45.856950 29678 rpcz_store.cc:276] 0320 15:27:40.521719 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.524569 (+ 2850us) service_pool.cc:225] Handling call
0320 15:27:45.856070 (+5331501us) ts_tablet_manager.cc:1151] Deleting tablet 3ed5d220ec4b406aa5bae970f785de63
0320 15:27:45.856086 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.856816 (+ 730us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5329695}
I20260320 15:27:45.857414 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 254) took 4998 ms. Trace:
I20260320 15:27:45.857503 29678 rpcz_store.cc:276] 0320 15:27:40.858557 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.858771 (+ 214us) service_pool.cc:225] Handling call
0320 15:27:45.857256 (+4998485us) ts_tablet_manager.cc:1151] Deleting tablet e779a274913f4caeaf82e3f180457830
0320 15:27:45.857270 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.857397 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4998265}
I20260320 15:27:45.857821 29678 tablet_replica.cc:333] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:45.858079 29678 raft_consensus.cc:2243] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:45.840279 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because the tablet was not found. No further retry: Not found: Tablet not found: 9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:45.858412 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:45.858882 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 89425da569794a8fa972817023bf853d because the tablet was not found. No further retry: Not found: Tablet not found: 89425da569794a8fa972817023bf853d
I20260320 15:27:45.858436 29678 raft_consensus.cc:2272] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.863437 30466 raft_consensus.cc:697] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:45.863898 30466 consensus_queue.cc:237] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.864481 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:45.864198 29678 ts_tablet_manager.cc:1916] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.867175 28736 tablet_bootstrap.cc:654] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:45.864732 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because the tablet was not found. No further retry: Not found: Tablet not found: 7c4b53fb135741efad5d3e59a6d4eb3e
W20260320 15:27:45.868136 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet a4bfb87d5d5741adb0ad2f2a88f4867f because the tablet was not found. No further retry: Not found: Tablet not found: a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:45.868402 28734 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:45.868743 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:45.868795 28734 ts_tablet_manager.cc:1403] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.169s user 0.004s sys 0.002s
I20260320 15:27:45.868999 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:45.869585 30155 raft_consensus.cc:493] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.869704 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:45.869849 30155 raft_consensus.cc:515] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.870380 28734 raft_consensus.cc:359] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.870559 28734 raft_consensus.cc:385] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.870605 28734 raft_consensus.cc:740] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.870762 30155 leader_election.cc:290] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.870972 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.871577 28734 consensus_queue.cc:260] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.870386 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:45.872519 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:45.872882 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
W20260320 15:27:45.872907 28382 leader_election.cc:343] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:45.873188 29640 ts_tablet_manager.cc:1950] T 27ccc693e7704650bd77e0c4ca11862e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:45.873497 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:45.874580 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 259) took 5491 ms. Trace:
I20260320 15:27:45.873955 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:45.876386 28733 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:45.876461 28524 leader_election.cc:304] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.876674 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:45.878134 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 171) took 10203 ms. Trace:
I20260320 15:27:45.878005 30395 raft_consensus.cc:2804] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.878448 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:45.878003 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.879158 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.880352 28733 ts_tablet_manager.cc:1403] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.163s user 0.011s sys 0.000s
I20260320 15:27:45.880614 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:45.881047 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:45.883812 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:45.884792 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:45.885309 28905 catalog_manager.cc:5671] T c7e3a6fa7dc74de286177a285dd3d877 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:45.886864 28734 ts_tablet_manager.cc:1434] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20260320 15:27:45.887370 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:45.887715 28734 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:45.888434 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:45.889381 28734 tablet_bootstrap.cc:654] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:45.889778 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:45.891040 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:45.891342 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:45.891812 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:45.899803 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:45.875253 29643 rpcz_store.cc:276] 0320 15:27:40.383049 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.383182 (+ 133us) service_pool.cc:225] Handling call
0320 15:27:45.874316 (+5491134us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:45.874331 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.874548 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5490988}
W20260320 15:27:45.901757 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because the tablet was not found. No further retry: Not found: Tablet not found: f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:45.902488 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:45.903291 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:45.903398 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 261) took 5520 ms. Trace:
I20260320 15:27:45.903466 28424 tablet_service.cc:1511] Processing CreateTablet for tablet ad6d3aae8e75428c90ed2b3265316f57 (DEFAULT_TABLE table=default.Table_467ca42537f0421bacf771620a950e06 [id=384735f1b98642b2b325bd31d66ee41d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.903519 29643 rpcz_store.cc:276] 0320 15:27:40.383258 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.383352 (+ 94us) service_pool.cc:225] Handling call
0320 15:27:45.903287 (+5519935us) ts_tablet_manager.cc:1151] Deleting tablet df5ffaedeaf94def93d93a538c42e3bd
0320 15:27:45.903300 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.903379 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5519778}
I20260320 15:27:45.903949 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6d3aae8e75428c90ed2b3265316f57. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:45.903985 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet df5ffaedeaf94def93d93a538c42e3bd because the tablet was not found. No further retry: Not found: Tablet not found: df5ffaedeaf94def93d93a538c42e3bd
W20260320 15:27:45.899173 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 65590.9us
I20260320 15:27:45.904940 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 263) took 5521 ms. Trace:
I20260320 15:27:45.905041 29643 rpcz_store.cc:276] 0320 15:27:40.383471 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.383578 (+ 107us) service_pool.cc:225] Handling call
0320 15:27:45.904838 (+5521260us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:45.904848 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.904921 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5521081}
W20260320 15:27:45.906551 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 1c2d7e4d26994a17bf2d2615d6552412 because the tablet was not found. No further retry: Not found: Tablet not found: 1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:45.907189 28905 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:45.907435 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.907644 28441 raft_consensus.cc:2468] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:45.908046 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.902274 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bb680bb0863485aaad85a749221e81b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.910820 29640 rpcz_store.cc:276] 0320 15:27:35.671041 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.671103 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:45.722467 (+10051364us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:45.722478 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.791236 (+ 68758us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.794763 (+ 3527us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.874205 (+ 79442us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10051140}
I20260320 15:27:45.911574 28574 raft_consensus.cc:2468] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:45.911696 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41da3cb06b1147fbae2695bc5c9ffaa5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:45.911866 30618 tablet.cc:1723] T 23eda54577404cc6900cb3ae348604d4 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:45.912276 28382 leader_election.cc:304] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.912706 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 52a9b7b73bc04a7bb937d863c375b10f (DEFAULT_TABLE table=default.Table_5228be23ce9740aa912d56fce56ec7ed [id=e4a61dea95f14ac09c99dfd92c90ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.912801 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 265) took 5529 ms. Trace:
I20260320 15:27:45.913524 30454 raft_consensus.cc:2804] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.913687 30454 raft_consensus.cc:493] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.913825 30395 raft_consensus.cc:493] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.913846 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 6725124b62ad4171a2cfe072687b8c3d (DEFAULT_TABLE table=default.Table_a3cd321472b34384aadb6e966f5402da [id=65f2ef6dd1884f6e974f3b1c77e1b567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.914045 30394 consensus_queue.cc:1048] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260320 15:27:45.914151 30454 raft_consensus.cc:3060] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.914799 29640 tablet_replica.cc:333] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:45.915357 30395 raft_consensus.cc:3060] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.916054 29640 raft_consensus.cc:2243] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:45.916165 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a9b7b73bc04a7bb937d863c375b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.916353 29640 raft_consensus.cc:2272] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:45.917039 29640 ts_tablet_manager.cc:1916] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:45.918473 30394 raft_consensus.cc:493] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.918987 30394 raft_consensus.cc:515] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.920295 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6725124b62ad4171a2cfe072687b8c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.920233 28733 raft_consensus.cc:359] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.921451 28733 raft_consensus.cc:385] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:45.921576 28733 raft_consensus.cc:740] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:45.921700 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e (DEFAULT_TABLE table=default.Table_786940b8acbe4d13944ef32c8b08cbba [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.922636 28733 consensus_queue.cc:260] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.923434 28733 ts_tablet_manager.cc:1434] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.043s user 0.002s sys 0.000s
I20260320 15:27:45.912879 29643 rpcz_store.cc:276] 0320 15:27:40.383741 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.383809 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:45.912690 (+5528881us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:45.912703 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.912783 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5528682}
I20260320 15:27:45.926987 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bd68c6e4364a038a815d1fc05ad25e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.927081 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 262) took 5543 ms. Trace:
I20260320 15:27:45.927151 29643 rpcz_store.cc:276] 0320 15:27:40.383352 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.384030 (+ 678us) service_pool.cc:225] Handling call
0320 15:27:45.926950 (+5542920us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:45.926961 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.927063 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5542789}
I20260320 15:27:45.927410 29643 tablet_replica.cc:333] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:45.927665 29643 raft_consensus.cc:2243] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:45.928009 29643 raft_consensus.cc:2272] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:45.928568 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 89425da569794a8fa972817023bf853d because the tablet was not found. No further retry: Not found: Tablet not found: 89425da569794a8fa972817023bf853d
I20260320 15:27:45.928814 29643 ts_tablet_manager.cc:1916] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:45.928985 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because the tablet was not found. No further retry: Not found: Tablet not found: 583ed053b2a742c7b029f348f4495101
I20260320 15:27:45.935019 29939 raft_consensus.cc:493] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.511312 29736 rpcz_store.cc:276] 0320 15:27:34.960633 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:34.960704 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:44.286507 (+9325803us) raft_consensus.cc:1471] Updating replica for 62 ops
0320 15:27:44.457409 (+170902us) raft_consensus.cc:1534] Early marking committed up to index 547
0320 15:27:44.540202 (+ 82793us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.540211 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:44.540564 (+ 353us) raft_consensus.cc:1539] Triggering prepare for 62 ops
0320 15:27:44.541033 (+ 469us) write_op.cc:270] Start()
0320 15:27:44.541086 (+ 53us) write_op.cc:276] Timestamp: P: 1774020450050142 usec, L: 0
0320 15:27:44.541496 (+ 410us) write_op.cc:270] Start()
0320 15:27:44.541524 (+ 28us) write_op.cc:276] Timestamp: P: 1774020450085670 usec, L: 0
0320 15:27:44.542246 (+ 722us) write_op.cc:270] Start()
0320 15:27:44.542274 (+ 28us) write_op.cc:276] Timestamp: P: 1774020450087693 usec, L: 0
0320 15:27:44.542431 (+ 157us) write_op.cc:270] Start()
0320 15:27:44.542452 (+ 21us) write_op.cc:276] Timestamp: P: 1774020450091672 usec, L: 0
0320 15:27:44.542607 (+ 155us) write_op.cc:270] Start()
0320 15:27:44.542627 (+ 20us) write_op.cc:276] Timestamp: P: 1774020450146883 usec, L: 0
0320 15:27:44.542767 (+ 140us) write_op.cc:270] Start()
0320 15:27:44.542786 (+ 19us) write_op.cc:276] Timestamp: P: 1774020450175091 usec, L: 0
0320 15:27:44.542991 (+ 205us) write_op.cc:270] Start()
0320 15:27:44.543092 (+ 101us) write_op.cc:276] Timestamp: P: 1774020450176540 usec, L: 0
0320 15:27:44.543273 (+ 181us) write_op.cc:270] Start()
0320 15:27:44.543295 (+ 22us) write_op.cc:276] Timestamp: P: 1774020450179442 usec, L: 0
0320 15:27:44.543462 (+ 167us) write_op.cc:270] Start()
0320 15:27:44.543485 (+ 23us) write_op.cc:276] Timestamp: P: 1774020450180681 usec, L: 0
0320 15:27:44.543666 (+ 181us) write_op.cc:270] Start()
0320 15:27:44.543692 (+ 26us) write_op.cc:276] Timestamp: P: 1774020450181790 usec, L: 0
0320 15:27:44.543842 (+ 150us) write_op.cc:270] Start()
0320 15:27:44.543862 (+ 20us) write_op.cc:276] Timestamp: P: 1774020450319041 usec, L: 0
0320 15:27:44.544686 (+ 824us) write_op.cc:270] Start()
0320 15:27:44.544717 (+ 31us) write_op.cc:276] Timestamp: P: 1774020450350255 usec, L: 0
0320 15:27:44.544918 (+ 201us) write_op.cc:270] Start()
0320 15:27:44.544943 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450380035 usec, L: 0
0320 15:27:44.545110 (+ 167us) write_op.cc:270] Start()
0320 15:27:44.545134 (+ 24us) write_op.cc:276] Timestamp: P: 1774020450390978 usec, L: 0
0320 15:27:44.545332 (+ 198us) write_op.cc:270] Start()
0320 15:27:44.545357 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450414329 usec, L: 0
0320 15:27:44.545541 (+ 184us) write_op.cc:270] Start()
0320 15:27:44.545566 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450422929 usec, L: 0
0320 15:27:44.545746 (+ 180us) write_op.cc:270] Start()
0320 15:27:44.545778 (+ 32us) write_op.cc:276] Timestamp: P: 1774020450448692 usec, L: 0
0320 15:27:44.545933 (+ 155us) write_op.cc:270] Start()
0320 15:27:44.545960 (+ 27us) write_op.cc:276] Timestamp: P: 1774020450457090 usec, L: 0
0320 15:27:44.546124 (+ 164us) write_op.cc:270] Start()
0320 15:27:44.546149 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450465315 usec, L: 0
0320 15:27:44.546292 (+ 143us) write_op.cc:270] Start()
0320 15:27:44.546312 (+ 20us) write_op.cc:276] Timestamp: P: 1774020450472281 usec, L: 0
0320 15:27:44.546477 (+ 165us) write_op.cc:270] Start()
0320 15:27:44.546502 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450628344 usec, L: 0
0320 15:27:44.546659 (+ 157us) write_op.cc:270] Start()
0320 15:27:44.546683 (+ 24us) write_op.cc:276] Timestamp: P: 1774020450629629 usec, L: 0
0320 15:27:44.546845 (+ 162us) write_op.cc:270] Start()
0320 15:27:44.546912 (+ 67us) write_op.cc:276] Timestamp: P: 1774020450665848 usec, L: 0
0320 15:27:44.547117 (+ 205us) write_op.cc:270] Start()
0320 15:27:44.547142 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450670996 usec, L: 0
0320 15:27:44.567359 (+ 20217us) write_op.cc:270] Start()
0320 15:27:44.567408 (+ 49us) write_op.cc:276] Timestamp: P: 1774020450682618 usec, L: 0
0320 15:27:44.567694 (+ 286us) write_op.cc:270] Start()
0320 15:27:44.567723 (+ 29us) write_op.cc:276] Timestamp: P: 1774020450738483 usec, L: 0
0320 15:27:44.567887 (+ 164us) write_op.cc:270] Start()
0320 15:27:44.567911 (+ 24us) write_op.cc:276] Timestamp: P: 1774020450755719 usec, L: 0
0320 15:27:44.568310 (+ 399us) write_op.cc:270] Start()
0320 15:27:44.568335 (+ 25us) write_op.cc:276] Timestamp: P: 1774020450775461 usec, L: 0
0320 15:27:44.568500 (+ 165us) write_op.cc:270] Start()
0320 15:27:44.568526 (+ 26us) write_op.cc:276] Timestamp: P: 1774020450925570 usec, L: 0
0320 15:27:44.568695 (+ 169us) write_op.cc:270] Start()
0320 15:27:44.568718 (+ 23us) write_op.cc:276] Timestamp: P: 1774020450993129 usec, L: 0
0320 15:27:44.568877 (+ 159us) write_op.cc:270] Start()
0320 15:27:44.568900 (+ 23us) write_op.cc:276] Timestamp: P: 1774020450994971 usec, L: 0
0320 15:27:44.569061 (+ 161us) write_op.cc:270] Start()
0320 15:27:44.569084 (+ 23us) write_op.cc:276] Timestamp: P: 1774020450998597 usec, L: 0
0320 15:27:44.569246 (+ 162us) write_op.cc:270] Start()
0320 15:27:44.569269 (+ 23us) write_op.cc:276] Timestamp: P: 1774020451066710 usec, L: 0
0320 15:27:44.569405 (+ 136us) write_op.cc:270] Start()
0320 15:27:44.569430 (+ 25us) write_op.cc:276] Timestamp: P: 1774020451115998 usec, L: 0
0320 15:27:44.569584 (+ 154us) write_op.cc:270] Start()
0320 15:27:44.569605 (+ 21us) write_op.cc:276] Timestamp: P: 1774020451837501 usec, L: 0
0320 15:27:44.621419 (+ 51814us) write_op.cc:270] Start()
0320 15:27:44.621491 (+ 72us) write_op.cc:276] Timestamp: P: 1774020452735578 usec, L: 0
0320 15:27:44.621782 (+ 291us) write_op.cc:270] Start()
0320 15:27:44.621822 (+ 40us) write_op.cc:276] Timestamp: P: 1774020452905081 usec, L: 0
0320 15:27:44.628870 (+ 7048us) write_op.cc:270] Start()
0320 15:27:44.628931 (+ 61us) write_op.cc:276] Timestamp: P: 1774020452914645 usec, L: 0
0320 15:27:44.629213 (+ 282us) write_op.cc:270] Start()
0320 15:27:44.629243 (+ 30us) write_op.cc:276] Timestamp: P: 1774020453038951 usec, L: 0
0320 15:27:44.629419 (+ 176us) write_op.cc:270] Start()
0320 15:27:44.629441 (+ 22us) write_op.cc:276] Timestamp: P: 1774020453103721 usec, L: 0
0320 15:27:44.629574 (+ 133us) write_op.cc:270] Start()
0320 15:27:44.629593 (+ 19us) write_op.cc:276] Timestamp: P: 1774020453232074 usec, L: 0
0320 15:27:44.629725 (+ 132us) write_op.cc:270] Start()
0320 15:27:44.629746 (+ 21us) write_op.cc:276] Timestamp: P: 1774020453442134 usec, L: 0
0320 15:27:44.629889 (+ 143us) write_op.cc:270] Start()
0320 15:27:44.629907 (+ 18us) write_op.cc:276] Timestamp: P: 1774020453636108 usec, L: 0
0320 15:27:44.630033 (+ 126us) write_op.cc:270] Start()
0320 15:27:44.630050 (+ 17us) write_op.cc:276] Timestamp: P: 1774020453686604 usec, L: 0
0320 15:27:44.630171 (+ 121us) write_op.cc:270] Start()
0320 15:27:44.630188 (+ 17us) write_op.cc:276] Timestamp: P: 1774020453694687 usec, L: 0
0320 15:27:44.630320 (+ 132us) write_op.cc:270] Start()
0320 15:27:44.630338 (+ 18us) write_op.cc:276] Timestamp: P: 1774020453695554 usec, L: 0
0320 15:27:44.630554 (+ 216us) write_op.cc:270] Start()
0320 15:27:44.630574 (+ 20us) write_op.cc:276] Timestamp: P: 1774020453733440 usec, L: 0
0320 15:27:44.630757 (+ 183us) write_op.cc:270] Start()
0320 15:27:44.630776 (+ 19us) write_op.cc:276] Timestamp: P: 1774020453759343 usec, L: 0
0320 15:27:44.630925 (+ 149us) write_op.cc:270] Start()
0320 15:27:44.630946 (+ 21us) write_op.cc:276] Timestamp: P: 1774020453790776 usec, L: 0
0320 15:27:44.631099 (+ 153us) write_op.cc:270] Start()
0320 15:27:44.631118 (+ 19us) write_op.cc:276] Timestamp: P: 1774020453996738 usec, L: 0
0320 15:27:44.631251 (+ 133us) write_op.cc:270] Start()
0320 15:27:44.631272 (+ 21us) write_op.cc:276] Timestamp: P: 1774020454119040 usec, L: 0
0320 15:27:44.631412 (+ 140us) write_op.cc:270] Start()
0320 15:27:44.631436 (+ 24us) write_op.cc:276] Timestamp: P: 1774020454261908 usec, L: 0
0320 15:27:44.659683 (+ 28247us) write_op.cc:270] Start()
0320 15:27:44.659736 (+ 53us) write_op.cc:276] Timestamp: P: 1774020454334410 usec, L: 0
0320 15:27:44.661600 (+ 1864us) write_op.cc:270] Start()
0320 15:27:44.661646 (+ 46us) write_op.cc:276] Timestamp: P: 1774020454335308 usec, L: 0
0320 15:27:44.661863 (+ 217us) write_op.cc:270] Start()
0320 15:27:44.661884 (+ 21us) write_op.cc:276] Timestamp: P: 1774020454335933 usec, L: 0
0320 15:27:44.662035 (+ 151us) write_op.cc:270] Start()
0320 15:27:44.662058 (+ 23us) write_op.cc:276] Timestamp: P: 1774020454372217 usec, L: 0
0320 15:27:44.662207 (+ 149us) write_op.cc:270] Start()
0320 15:27:44.662227 (+ 20us) write_op.cc:276] Timestamp: P: 1774020454372977 usec, L: 0
0320 15:27:44.662371 (+ 144us) write_op.cc:270] Start()
0320 15:27:44.662390 (+ 19us) write_op.cc:276] Timestamp: P: 1774020454421188 usec, L: 0
0320 15:27:44.662554 (+ 164us) write_op.cc:270] Start()
0320 15:27:44.662574 (+ 20us) write_op.cc:276] Timestamp: P: 1774020454515409 usec, L: 0
0320 15:27:44.665377 (+ 2803us) write_op.cc:270] Start()
0320 15:27:44.665417 (+ 40us) write_op.cc:276] Timestamp: P: 1774020454532131 usec, L: 0
0320 15:27:44.673790 (+ 8373us) write_op.cc:270] Start()
0320 15:27:44.673840 (+ 50us) write_op.cc:276] Timestamp: P: 1774020454696717 usec, L: 0
0320 15:27:44.681973 (+ 8133us) log.cc:844] Serialized 53895 byte log entry
0320 15:27:44.706224 (+ 24251us) raft_consensus.cc:1645] Marking committed up to 609
0320 15:27:44.706470 (+ 246us) log.cc:844] Serialized 15 byte log entry
0320 15:27:44.851124 (+144654us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.851133 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:44.852214 (+ 1081us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.852220 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:44.852909 (+ 689us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.852924 (+ 15us) write_op.cc:454] Released schema lock
0320 15:27:44.853591 (+ 667us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.853596 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.854254 (+ 658us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.854259 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.854998 (+ 739us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.855003 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.855724 (+ 721us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.855729 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.856287 (+ 558us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.856291 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:44.856821 (+ 530us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.856826 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.857383 (+ 557us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.857388 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.883634 (+ 26246us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.883665 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:44.909435 (+ 25770us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.909444 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:44.920803 (+ 11359us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.920812 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:44.923133 (+ 2321us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.923138 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.926901 (+ 3763us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.926906 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:44.928323 (+ 1417us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.928327 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:44.928976 (+ 649us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.928979 (+ 3us) write_op.cc:454] Released schema lock
0320 15:27:44.929578 (+ 599us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.929582 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:44.934139 (+ 4557us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.934146 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:45.010802 (+ 76656us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.010812 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:45.023052 (+ 12240us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.023061 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:45.074723 (+ 51662us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.074733 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:45.116341 (+ 41608us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.116353 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:45.153213 (+ 36860us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.153229 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:45.194391 (+ 41162us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.194397 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:45.224780 (+ 30383us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.224801 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:45.289935 (+ 65134us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.289941 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:45.318113 (+ 28172us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.318126 (+ 13us) write_op.cc:454] Released schema lock
0320 15:27:45.439867 (+121741us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.439924 (+ 57us) write_op.cc:454] Released schema lock
0320 15:27:45.441724 (+ 1800us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.441732 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:45.447967 (+ 6235us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.447975 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:45.484347 (+ 36372us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.484357 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:45.492426 (+ 8069us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.492436 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:45.494582 (+ 2146us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.494587 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:45.496608 (+ 2021us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.496614 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:45.499877 (+ 3263us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.499882 (+ 5us) write_op.cc:454] Released schema lock
0320 15:27:45.505085 (+ 5203us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.505089 (+ 4us) write_op.cc:454] Released schema lock
0320 15:27:45.510720 (+ 5631us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.510727 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:45.510802 (+ 75us) raft_consensus.cc:1697] Filling consensus response to leader.
0320 15:27:45.510861 (+ 59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0320 15:27:45.510888 (+ 27us) raft_consensus.cc:1685] finished
0320 15:27:45.510893 (+ 5us) raft_consensus.cc:1691] UpdateReplicas() finished
0320 15:27:45.511053 (+ 160us) inbound_call.cc:173] Queueing success response
0320 15:27:45.511625 (+ 572us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x616000060a54. stack: 00007fd170a38076 00007fd170a37e30 00007fd16f9f23ad 0000000000986fb4 0000000000986f7d 00007fd17fd786bd 00007fd17fd787b3 00007fd17c93ab21 00007fd184803755 00007fd1848175b5 00007fd184817361 00007fd17dcb97dd 00007fd17c94b92e 00007fd17c94b67d 0000000000a8b58c 00007fd170a7df5f
0320 15:27:45.512778 (+ 1153us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x616000060a54. stack: 00007fd170a38076 00007fd170a37e30 00007fd16f9f23ad 0000000000986fb4 0000000000986f7d 00007fd17fd786bd 00007fd17fd787b3 00007fd17c93ab21 00007fd184803755 00007fd1848175b5 00007fd184817361 00007fd17dcb97dd 00007fd17c94b92e 00007fd17c94b67d 0000000000a8b58c 00007fd170a7df5f
Related trace 'op':
0320 15:27:44.542238 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.542384 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.542389 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.542392 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.542499 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.542516 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.542521 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.542586 (+ 65us) tablet.cc:700] Row locks acquired
0320 15:27:44.542589 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.849701 (+307112us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.849851 (+ 150us) write_op.cc:301] APPLY: starting
0320 15:27:44.849872 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.849931 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.849934 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.850065 (+ 131us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.850103 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.850174 (+ 71us) write_op.cc:312] APPLY: finished
0320 15:27:44.850313 (+ 139us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.850457 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.850494 (+ 37us) write_op.cc:454] Released schema lock
0320 15:27:44.850503 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.542666 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.542760 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.542765 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.542768 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:44.542934 (+ 166us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.542951 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.542958 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.543027 (+ 69us) tablet.cc:700] Row locks acquired
0320 15:27:44.543030 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.851232 (+308202us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.851319 (+ 87us) write_op.cc:301] APPLY: starting
0320 15:27:44.851332 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.851374 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.851377 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.851496 (+ 119us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.851598 (+ 102us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.851641 (+ 43us) write_op.cc:312] APPLY: finished
0320 15:27:44.851719 (+ 78us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.851785 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.851801 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:44.851807 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.543093 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.543183 (+ 90us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.543188 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.543190 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.543273 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.543284 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.543289 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.543356 (+ 67us) tablet.cc:700] Row locks acquired
0320 15:27:44.543359 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.852319 (+308960us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.852391 (+ 72us) write_op.cc:301] APPLY: starting
0320 15:27:44.852402 (+ 11us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.852436 (+ 34us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.852438 (+ 2us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.852479 (+ 41us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.852507 (+ 28us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.852547 (+ 40us) write_op.cc:312] APPLY: finished
0320 15:27:44.852614 (+ 67us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.852673 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.852685 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:44.852690 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.543425 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.543559 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.543564 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.543566 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.543694 (+ 128us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.543710 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.543717 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.543798 (+ 81us) tablet.cc:700] Row locks acquired
0320 15:27:44.543802 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.852987 (+309185us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.853050 (+ 63us) write_op.cc:301] APPLY: starting
0320 15:27:44.853060 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.853090 (+ 30us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.853093 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.853136 (+ 43us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.853162 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:44.853212 (+ 50us) write_op.cc:312] APPLY: finished
0320 15:27:44.853273 (+ 61us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.853338 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.853353 (+ 15us) write_op.cc:454] Released schema lock
0320 15:27:44.853358 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.543877 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.543983 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.543989 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:44.543994 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:44.544068 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.544082 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.544089 (+ 7us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:44.544137 (+ 48us) tablet.cc:700] Row locks acquired
0320 15:27:44.544141 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:44.853711 (+309570us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.853788 (+ 77us) write_op.cc:301] APPLY: starting
0320 15:27:44.853798 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.853824 (+ 26us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.853826 (+ 2us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.853858 (+ 32us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.853881 (+ 23us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.853916 (+ 35us) write_op.cc:312] APPLY: finished
0320 15:27:44.853969 (+ 53us) log.cc:844] Serialized 23 byte log entry
0320 15:27:44.854022 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.854034 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:44.854040 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.544248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.544323 (+ 75us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.544328 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.544330 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.544391 (+ 61us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.544400 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.544404 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.544450 (+ 46us) tablet.cc:700] Row locks acquired
0320 15:27:44.544453 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:44.854330 (+309877us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.854411 (+ 81us) write_op.cc:301] APPLY: starting
0320 15:27:44.854421 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.854465 (+ 44us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.854468 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.854564 (+ 96us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.854613 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.854649 (+ 36us) write_op.cc:312] APPLY: finished
0320 15:27:44.854709 (+ 60us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.854761 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.854775 (+ 14us) write_op.cc:454] Released schema lock
0320 15:27:44.854781 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.544501 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.544563 (+ 62us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.544566 (+ 3us) write_op.cc:435] Acquired schema lock
0320 15:27:44.544568 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.544659 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.544668 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:44.544672 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:44.544720 (+ 48us) tablet.cc:700] Row locks acquired
0320 15:27:44.544722 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:44.855092 (+310370us) op_driver.cc:464] REPLICATION: finished
0320 15:27:44.855158 (+ 66us) write_op.cc:301] APPLY: starting
0320 15:27:44.855168 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:44.855195 (+ 27us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:44.855198 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:44.855301 (+ 103us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:44.855349 (+ 48us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:44.855370 (+ 21us) write_op.cc:312] APPLY: finished
0320 15:27:44.855427 (+ 57us) log.cc:844] Serialized 29 byte log entry
0320 15:27:44.855479 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:44.855491 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:44.855496 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:44.544772 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:44.544839 (+ 67us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:44.544844 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:44.544846 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:44.544906 (+ 60us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:44.544914 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0320 15:27
I20260320 15:27:45.937960 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 27ccc693e7704650bd77e0c4ca11862e (table default.Table_3787678c407745418136143b1e78fd61 [id=d4e0e4c8f591402a9aa97bd21e763ed1]) successfully deleted
I20260320 15:27:45.939518 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ad6d3aae8e75428c90ed2b3265316f57 (DEFAULT_TABLE table=default.Table_467ca42537f0421bacf771620a950e06 [id=384735f1b98642b2b325bd31d66ee41d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.941256 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 12c67b9c53b049588282141c75a61084 (DEFAULT_TABLE table=default.Table_164db1bde0574cd5ad200b6c2c4bb557 [id=2d0aebd58efa4645bf1dbc80fd5e5b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.941871 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c67b9c53b049588282141c75a61084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.942237 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.942648 30454 raft_consensus.cc:515] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.943382 30454 leader_election.cc:290] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.946511 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.946745 28444 raft_consensus.cc:2468] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:45.946771 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.950312 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 4f7d26bc629d4b6d852c7e966cd7bc45 (DEFAULT_TABLE table=default.Table_488a72bf7d2e4ab0a4c050fdb1e707c5 [id=ffbf882caf9f4f38aaabe9633987e1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.951062 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f7d26bc629d4b6d852c7e966cd7bc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:45.952519 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.952594 30623 raft_consensus.cc:493] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:45.952700 30623 raft_consensus.cc:515] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.953328 30623 leader_election.cc:290] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:45.957964 30587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:45.958431 30587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
I20260320 15:27:45.958963 29678 ts_tablet_manager.cc:1929] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:45.964141 28551 tablet_service.cc:1511] Processing CreateTablet for tablet 4f7d26bc629d4b6d852c7e966cd7bc45 (DEFAULT_TABLE table=default.Table_488a72bf7d2e4ab0a4c050fdb1e707c5 [id=ffbf882caf9f4f38aaabe9633987e1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:45.965292 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet a185f84c24ea402684aa1cc3628d4f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.969954 29681 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "80636bb608a1431b955576e296ff63bb" instance_seqno: 1774020437674462) as {username='slave'} at 127.27.74.3:58673; Asking this server to re-register.
I20260320 15:27:45.971611 28733 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:45.974205 28524 leader_election.cc:304] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:45.959096 29678 log.cc:1199] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/35347116811e4aee8142a503dd21843a
I20260320 15:27:45.978691 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:45.979835 28902 catalog_manager.cc:5671] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:45.980839 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:45.981379 29939 raft_consensus.cc:515] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:45.982153 29939 leader_election.cc:290] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.982129 28902 catalog_manager.cc:5671] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:45.983095 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:45.984041 30155 consensus_queue.cc:1048] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260320 15:27:45.984601 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.984954 30394 leader_election.cc:290] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:45.985266 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.985589 29678 ts_tablet_manager.cc:1950] T 35347116811e4aee8142a503dd21843a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:45.985791 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:45.986470 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 258) took 4493 ms. Trace:
I20260320 15:27:45.986554 29678 rpcz_store.cc:276] 0320 15:27:41.493158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.493219 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:45.857764 (+4364545us) ts_tablet_manager.cc:1151] Deleting tablet 35347116811e4aee8142a503dd21843a
0320 15:27:45.857775 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.905080 (+ 47305us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.958827 (+ 53747us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:45.986446 (+ 27619us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4364322}
I20260320 15:27:45.987110 28902 catalog_manager.cc:5671] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:45.988828 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:45.989126 30619 raft_consensus.cc:2804] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:45.989296 30619 raft_consensus.cc:493] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:45.989454 30619 raft_consensus.cc:3060] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:45.989643 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:45.990449 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:45.992889 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 259) took 4449 ms. Trace:
I20260320 15:27:45.993013 29678 rpcz_store.cc:276] 0320 15:27:41.543058 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.543109 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:45.986829 (+4443720us) ts_tablet_manager.cc:1151] Deleting tablet 87cb35cefee841d0881b08c456df5a97
0320 15:27:45.986841 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.992857 (+ 6016us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4443517}
I20260320 15:27:45.993624 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 262) took 3452 ms. Trace:
I20260320 15:27:45.994964 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:45.997347 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:45.997642 28440 raft_consensus.cc:2393] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:45.994011 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.002023 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:46.001720 29678 rpcz_store.cc:276] 0320 15:27:42.540989 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.541181 (+ 192us) service_pool.cc:225] Handling call
0320 15:27:45.993472 (+3452291us) ts_tablet_manager.cc:1151] Deleting tablet aba79ddde6e842dfb67597c5d9e371cb
0320 15:27:45.993486 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.993608 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3452064}
I20260320 15:27:46.005396 28383 leader_election.cc:304] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.005730 30454 raft_consensus.cc:2749] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:46.006088 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 191772us
I20260320 15:27:46.007797 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 263) took 2820 ms. Trace:
I20260320 15:27:46.009128 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:46.009310 29678 rpcz_store.cc:276] 0320 15:27:43.187927 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.188021 (+ 94us) service_pool.cc:225] Handling call
0320 15:27:46.005526 (+2817505us) ts_tablet_manager.cc:1151] Deleting tablet 2e9791ac61b34fc09a8292c79f0c4032
0320 15:27:46.005809 (+ 283us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.007375 (+ 1566us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2816981}
I20260320 15:27:46.015161 30454 raft_consensus.cc:493] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.015384 30454 raft_consensus.cc:515] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.016155 30454 leader_election.cc:290] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:46.016884 28383 leader_election.cc:343] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.017172 28383 leader_election.cc:304] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.017556 30456 raft_consensus.cc:2749] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.017793 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:46.017669 28280 tablet_service.cc:1511] Processing CreateTablet for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (DEFAULT_TABLE table=default.Table_76bddc5536ff43e895889c5c30c5068f [id=7be232d8d79846eb858e6dd529427db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.016732 29678 tablet_replica.cc:333] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:46.019676 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8ac11b0ca134c948fdd9ff4b114e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.021265 29643 ts_tablet_manager.cc:1929] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260320 15:27:46.021396 29643 log.cc:1199] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:46.020751 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:46.021766 29643 ts_tablet_manager.cc:1950] T 9dbd3b0e92a744dcbc132037c67303ab P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.022586 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 268) took 5638 ms. Trace:
I20260320 15:27:46.022686 29643 rpcz_store.cc:276] 0320 15:27:40.384132 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.384182 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:45.927362 (+5543180us) ts_tablet_manager.cc:1151] Deleting tablet 9dbd3b0e92a744dcbc132037c67303ab
0320 15:27:45.927370 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.948331 (+ 20961us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.021148 (+ 72817us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.022562 (+ 1414us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5542996}
I20260320 15:27:46.023217 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:46.023783 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 264) took 5640 ms. Trace:
I20260320 15:27:46.023875 29643 rpcz_store.cc:276] 0320 15:27:40.383621 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.384396 (+ 775us) service_pool.cc:225] Handling call
0320 15:27:46.023662 (+5639266us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:46.023675 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.023764 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5639141}
I20260320 15:27:46.024185 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 3226f3fe64c14ec6b0ff01fe5a99add2 (DEFAULT_TABLE table=default.Table_6a46b3258ceb4784a9b3ed584b7fcae1 [id=5b2a9b0165384ee2946a512198267872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.024370 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 266) took 5640 ms. Trace:
I20260320 15:27:46.024452 29643 rpcz_store.cc:276] 0320 15:27:40.383895 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.384541 (+ 646us) service_pool.cc:225] Handling call
0320 15:27:46.024270 (+5639729us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:46.024279 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.024351 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5639604}
I20260320 15:27:46.024657 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226f3fe64c14ec6b0ff01fe5a99add2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.024559 29678 raft_consensus.cc:2243] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.027515 28278 tablet_service.cc:1511] Processing CreateTablet for tablet 3d985575b862410792b016c53df4d139 (DEFAULT_TABLE table=default.Table_59eba3ec25134b28aeb775da692dae4b [id=268b3f3861da4c5c9e1691f87a0ea784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.028141 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d985575b862410792b016c53df4d139. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.027583 29678 raft_consensus.cc:2272] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.030879 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet fcff253b9cce45b4ad11dc38397afee2 with delete_type TABLET_DATA_DELETED (replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.034639 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet cc97a5f2bd964a05bb7b2427226debdd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a2003e2af9a4d9c854990e1db41af28) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.036094 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 260) took 5652 ms. Trace:
I20260320 15:27:46.036211 29643 rpcz_store.cc:276] 0320 15:27:40.383159 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.385298 (+ 2139us) service_pool.cc:225] Handling call
0320 15:27:46.035958 (+5650660us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:46.035968 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.036065 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5649236}
I20260320 15:27:46.038141 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 4190329df1b644c392ff8cbace9aae7a with delete_type TABLET_DATA_DELETED (replaced by tablet 806d635bb5594f0bb826b9b31468b485) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.039439 28734 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.039803 29678 ts_tablet_manager.cc:1916] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.040529 28734 ts_tablet_manager.cc:1403] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.153s user 0.003s sys 0.001s
I20260320 15:27:46.042083 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet d04ab44039124f36ac60601098586b4e with delete_type TABLET_DATA_DELETED (replaced by tablet f77a32615c954da581e9a9e4cb45c497) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.044416 28734 raft_consensus.cc:359] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.045420 28734 raft_consensus.cc:385] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.045907 28734 raft_consensus.cc:740] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.046324 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 6725124b62ad4171a2cfe072687b8c3d (DEFAULT_TABLE table=default.Table_a3cd321472b34384aadb6e966f5402da [id=65f2ef6dd1884f6e974f3b1c77e1b567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.046818 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6725124b62ad4171a2cfe072687b8c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.046875 30466 raft_consensus.cc:493] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.046999 30466 raft_consensus.cc:515] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.047659 30466 leader_election.cc:290] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.047392 28734 consensus_queue.cc:260] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.048312 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.048586 28441 raft_consensus.cc:2393] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:46.039783 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 257) took 5657 ms. Trace:
I20260320 15:27:46.049669 29643 rpcz_store.cc:276] 0320 15:27:40.382499 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.382666 (+ 167us) service_pool.cc:225] Handling call
0320 15:27:46.039659 (+5656993us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:46.039669 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.039764 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5650665}
I20260320 15:27:46.050367 28734 ts_tablet_manager.cc:1434] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260320 15:27:46.051398 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 52a9b7b73bc04a7bb937d863c375b10f (DEFAULT_TABLE table=default.Table_5228be23ce9740aa912d56fce56ec7ed [id=e4a61dea95f14ac09c99dfd92c90ffda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.051873 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52a9b7b73bc04a7bb937d863c375b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.052099 29643 tablet_replica.cc:333] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.053036 28734 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.053577 29643 raft_consensus.cc:2243] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.054563 29643 raft_consensus.cc:2272] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.054684 28268 tablet_service.cc:1511] Processing CreateTablet for tablet ad6d3aae8e75428c90ed2b3265316f57 (DEFAULT_TABLE table=default.Table_467ca42537f0421bacf771620a950e06 [id=384735f1b98642b2b325bd31d66ee41d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.055080 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6d3aae8e75428c90ed2b3265316f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.058379 29643 ts_tablet_manager.cc:1916] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.062017 28734 tablet_bootstrap.cc:654] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.065641 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 4f7d26bc629d4b6d852c7e966cd7bc45 (DEFAULT_TABLE table=default.Table_488a72bf7d2e4ab0a4c050fdb1e707c5 [id=ffbf882caf9f4f38aaabe9633987e1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.066090 30466 raft_consensus.cc:493] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.066250 30466 raft_consensus.cc:515] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.066720 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f7d26bc629d4b6d852c7e966cd7bc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.066951 30466 leader_election.cc:290] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.067734 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.068011 28441 raft_consensus.cc:2393] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:46.070902 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 12c67b9c53b049588282141c75a61084 (DEFAULT_TABLE table=default.Table_164db1bde0574cd5ad200b6c2c4bb557 [id=2d0aebd58efa4645bf1dbc80fd5e5b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.071866 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c67b9c53b049588282141c75a61084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.079005 28297 raft_consensus.cc:3060] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.083889 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e (DEFAULT_TABLE table=default.Table_786940b8acbe4d13944ef32c8b08cbba [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.085085 28734 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.085129 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bd68c6e4364a038a815d1fc05ad25e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.085453 28734 ts_tablet_manager.cc:1403] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.001s
I20260320 15:27:46.088150 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 12c67b9c53b049588282141c75a61084 (DEFAULT_TABLE table=default.Table_164db1bde0574cd5ad200b6c2c4bb557 [id=2d0aebd58efa4645bf1dbc80fd5e5b87]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:46.024376 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 8a18f32c29224bd883ceb20658dff228 because the tablet was not found. No further retry: Not found: Tablet not found: 8a18f32c29224bd883ceb20658dff228
W20260320 15:27:46.088891 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 64555.4us
I20260320 15:27:46.088003 28734 raft_consensus.cc:359] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:46.089540 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ffe278739bbc4ddab848af9c25241380 because the tablet was not found. No further retry: Not found: Tablet not found: ffe278739bbc4ddab848af9c25241380
W20260320 15:27:46.089816 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 26818e620ff649d18b671e4f4c7c7e74 because the tablet was not found. No further retry: Not found: Tablet not found: 26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:46.090037 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f7d26bc629d4b6d852c7e966cd7bc45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:46.090065 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b2355eadcae6483bae7ae475d74dd131 because the tablet was not found. No further retry: Not found: Tablet not found: b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:46.091414 28635 heartbeater.cc:461] Registering TS with master...
I20260320 15:27:46.091215 30395 raft_consensus.cc:515] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.092128 30395 leader_election.cc:290] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.092355 28582 raft_consensus.cc:3060] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.023710 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:46.095379 28582 raft_consensus.cc:2468] T 008621b6a3fc40149a1c6aff8b12ef4e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:46.095623 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e (DEFAULT_TABLE table=default.Table_786940b8acbe4d13944ef32c8b08cbba [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.095955 30155 raft_consensus.cc:493] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.096093 30155 raft_consensus.cc:515] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.096820 30155 leader_election.cc:290] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.089277 28734 raft_consensus.cc:385] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.097628 28734 raft_consensus.cc:740] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.097949 28734 consensus_queue.cc:260] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.098654 28734 ts_tablet_manager.cc:1434] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260320 15:27:46.101572 29643 ts_tablet_manager.cc:1929] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:46.101816 29643 log.cc:1199] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:46.102299 29643 ts_tablet_manager.cc:1950] T 3ed5d220ec4b406aa5bae970f785de63 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.103804 28382 leader_election.cc:304] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.104121 30454 raft_consensus.cc:2804] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.107184 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.108368 28382 leader_election.cc:343] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.107050 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:46.109053 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6d3aae8e75428c90ed2b3265316f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.109625 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55bd68c6e4364a038a815d1fc05ad25e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.109839 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12c67b9c53b049588282141c75a61084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.113233 30619 raft_consensus.cc:515] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:46.114511 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 135783us
I20260320 15:27:46.115289 30619 leader_election.cc:290] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.118108 29640 ts_tablet_manager.cc:1929] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.118813 28524 leader_election.cc:304] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.120468 30454 raft_consensus.cc:697] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:46.120893 30454 consensus_queue.cc:237] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.121745 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.121937 28441 raft_consensus.cc:2468] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.122979 28734 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.124385 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 23eda54577404cc6900cb3ae348604d4 (table default.Table_ab12be810262486ca2ec3234763b510b [id=bf25edfb9f0e4924aadf3e85e57fab77]). Expected version 1 got 0
I20260320 15:27:46.124665 28734 tablet_bootstrap.cc:654] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.124972 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:46.125578 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.125759 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_13967321267549daa281ef67538d1894 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 1 got 0
I20260320 15:27:46.125813 28297 raft_consensus.cc:2468] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.126000 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:46.126128 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 273) took 5603 ms. Trace:
I20260320 15:27:46.126240 29643 rpcz_store.cc:276] 0320 15:27:40.523117 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.523172 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:46.051711 (+5528539us) ts_tablet_manager.cc:1151] Deleting tablet 3ed5d220ec4b406aa5bae970f785de63
0320 15:27:46.051725 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.075604 (+ 23879us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.101426 (+ 25822us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.126103 (+ 24677us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5528059}
I20260320 15:27:46.126679 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 280) took 5457 ms. Trace:
I20260320 15:27:46.126761 29643 rpcz_store.cc:276] 0320 15:27:40.669375 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.669531 (+ 156us) service_pool.cc:225] Handling call
0320 15:27:46.126568 (+5457037us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:46.126578 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.126663 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5454912}
I20260320 15:27:46.126852 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]). Expected version 1 got 0
I20260320 15:27:46.127045 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 281) took 5425 ms. Trace:
I20260320 15:27:46.127147 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:46.127125 29643 rpcz_store.cc:276] 0320 15:27:40.701245 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.702923 (+ 1678us) service_pool.cc:225] Handling call
0320 15:27:46.126970 (+5424047us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:46.126978 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.127032 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5423186}
I20260320 15:27:46.127652 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 282) took 5269 ms. Trace:
I20260320 15:27:46.127728 29643 rpcz_store.cc:276] 0320 15:27:40.858682 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.859028 (+ 346us) service_pool.cc:225] Handling call
0320 15:27:46.127505 (+5268477us) ts_tablet_manager.cc:1151] Deleting tablet e779a274913f4caeaf82e3f180457830
0320 15:27:46.127516 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.127633 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5268293}
I20260320 15:27:46.127967 29643 tablet_replica.cc:333] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.128583 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 3ed5d220ec4b406aa5bae970f785de63 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]) successfully deleted
W20260320 15:27:46.128887 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7be03035dc3d44f9aa5b43b49281a940 because the tablet was not found. No further retry: Not found: Tablet not found: 7be03035dc3d44f9aa5b43b49281a940
W20260320 15:27:46.129115 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 533105ad6e054fe3a4db32855bc688bd because the tablet was not found. No further retry: Not found: Tablet not found: 533105ad6e054fe3a4db32855bc688bd
W20260320 15:27:46.129388 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e779a274913f4caeaf82e3f180457830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e779a274913f4caeaf82e3f180457830 already in progress: creating tablet
I20260320 15:27:46.130329 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.130518 28441 raft_consensus.cc:3060] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.131182 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.131366 28297 raft_consensus.cc:3060] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.133562 28441 raft_consensus.cc:2468] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.133953 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.134085 28297 raft_consensus.cc:2468] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.134162 28444 raft_consensus.cc:2393] T 9b838187c21448b6909ad183f9824d2b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:46.134424 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.134563 28446 raft_consensus.cc:3060] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.135869 28297 raft_consensus.cc:3060] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.137473 28446 raft_consensus.cc:2468] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.141263 29643 raft_consensus.cc:2243] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.141642 29643 raft_consensus.cc:2272] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.142567 29643 ts_tablet_manager.cc:1916] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.150090 28736 tablet_bootstrap.cc:492] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.150323 28736 ts_tablet_manager.cc:1403] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.296s user 0.008s sys 0.004s
I20260320 15:27:46.151999 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:46.152482 28902 catalog_manager.cc:5671] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:46.153364 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:46.153903 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:46.154425 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:46.154733 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:46.155642 29939 raft_consensus.cc:2804] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.155701 28524 leader_election.cc:304] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.155776 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:46.155810 29939 raft_consensus.cc:493] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.155941 29939 raft_consensus.cc:3060] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.151403 28736 raft_consensus.cc:359] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.156136 28736 raft_consensus.cc:385] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.156216 28736 raft_consensus.cc:740] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.156477 28736 consensus_queue.cc:260] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.156787 28524 leader_election.cc:304] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.157003 28736 ts_tablet_manager.cc:1434] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:46.157300 28736 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.157317 28524 leader_election.cc:304] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.157686 30395 raft_consensus.cc:2804] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.157735 30619 raft_consensus.cc:2804] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.157799 30395 raft_consensus.cc:493] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.157922 30395 raft_consensus.cc:3060] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.158248 30394 raft_consensus.cc:2804] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.158448 30394 raft_consensus.cc:697] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.158861 30394 consensus_queue.cc:237] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.161680 28734 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.158857 29939 raft_consensus.cc:515] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.162369 28734 ts_tablet_manager.cc:1403] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.000s
I20260320 15:27:46.163268 28736 tablet_bootstrap.cc:654] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.165230 29643 ts_tablet_manager.cc:1929] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:46.167475 29640 log.cc:1199] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6771bd6a3f0e47d99a0191eae25cb2d2
I20260320 15:27:46.165508 28734 raft_consensus.cc:359] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.167824 28734 raft_consensus.cc:385] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.167951 28734 raft_consensus.cc:740] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.168262 28734 consensus_queue.cc:260] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.168881 28734 ts_tablet_manager.cc:1434] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260320 15:27:46.169322 28734 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.166949 28902 catalog_manager.cc:5671] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:46.170711 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]). Expected version 1 got 0
I20260320 15:27:46.170904 29643 log.cc:1199] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/35347116811e4aee8142a503dd21843a
I20260320 15:27:46.170958 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:46.171241 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:46.171471 28902 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:46.171774 29643 ts_tablet_manager.cc:1950] T 35347116811e4aee8142a503dd21843a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.172127 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet d823ab41194542fba5540e53e9f4680b
W20260320 15:27:46.172232 28904 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 325caab9b8f34f04a97b41137652ccdb consensus state has no leader
I20260320 15:27:46.172462 28904 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 135) took 2246 ms. Trace:
I20260320 15:27:46.172853 28906 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 141) took 1241 ms. Trace:
I20260320 15:27:46.172991 28906 rpcz_store.cc:276] 0320 15:27:44.931800 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.729454 (+797654us) service_pool.cc:225] Handling call
0320 15:27:45.729997 (+ 543us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.730163 (+ 166us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:45.730257 (+ 94us) catalog_manager.cc:2824] Locking tablets
0320 15:27:45.739443 (+ 9186us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:45.760291 (+ 20848us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:45.760391 (+ 100us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:46.172064 (+411673us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:46.172764 (+ 700us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.740277 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.740401 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.740407 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:45.740410 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.740571 (+ 161us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.740584 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.740589 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.740730 (+ 141us) tablet.cc:700] Row locks acquired
0320 15:27:45.740732 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:45.740771 (+ 39us) write_op.cc:270] Start()
0320 15:27:45.740812 (+ 41us) write_op.cc:276] Timestamp: P: 1774020465740713 usec, L: 0
0320 15:27:45.740816 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.741024 (+ 208us) log.cc:844] Serialized 725 byte log entry
0320 15:27:45.753762 (+ 12738us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.754440 (+ 678us) write_op.cc:301] APPLY: starting
0320 15:27:45.754451 (+ 11us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.754482 (+ 31us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.754522 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.754567 (+ 45us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.754592 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.754598 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:45.754644 (+ 46us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.754677 (+ 33us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.754691 (+ 14us) write_op.cc:454] Released schema lock
0320 15:27:45.754695 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.754747 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.754750 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":649,"apply.run_cpu_time_us":302,"apply.run_wall_time_us":302,"num_ops":2,"prepare.queue_time_us":64,"prepare.run_cpu_time_us":825,"prepare.run_wall_time_us":826,"replication_time_us":12931}]]}
I20260320 15:27:46.175487 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:46.176084 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 286) took 4730 ms. Trace:
I20260320 15:27:46.176219 28892 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 142) took 1240 ms. Trace:
I20260320 15:27:46.176174 29643 rpcz_store.cc:276] 0320 15:27:41.445919 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.445993 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:46.127927 (+4681934us) ts_tablet_manager.cc:1151] Deleting tablet 35347116811e4aee8142a503dd21843a
0320 15:27:46.127935 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.151509 (+ 23574us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.165101 (+ 13592us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.176060 (+ 10959us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13029,"tablet-delete.queue_time_us":4681663}
I20260320 15:27:46.176951 28734 tablet_bootstrap.cc:654] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.176357 28892 rpcz_store.cc:276] 0320 15:27:44.935849 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.756528 (+820679us) service_pool.cc:225] Handling call
0320 15:27:45.757046 (+ 518us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.757203 (+ 157us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:45.757267 (+ 64us) catalog_manager.cc:2824] Locking tablets
0320 15:27:45.757401 (+ 134us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:45.768279 (+ 10878us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:45.768367 (+ 88us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:46.175402 (+407035us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:46.176142 (+ 740us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.758207 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.758352 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.758359 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:45.758362 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.758481 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.758498 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.758505 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.758632 (+ 127us) tablet.cc:700] Row locks acquired
0320 15:27:45.758635 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.758673 (+ 38us) write_op.cc:270] Start()
0320 15:27:45.758708 (+ 35us) write_op.cc:276] Timestamp: P: 1774020465758613 usec, L: 0
0320 15:27:45.758712 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.758911 (+ 199us) log.cc:844] Serialized 759 byte log entry
0320 15:27:45.765270 (+ 6359us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.765388 (+ 118us) write_op.cc:301] APPLY: starting
0320 15:27:45.765403 (+ 15us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.765445 (+ 42us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.765472 (+ 27us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.765548 (+ 76us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.765580 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.765587 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.765659 (+ 72us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.765715 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.765735 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:45.765746 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.765811 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.765814 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":415,"apply.run_wall_time_us":414,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":786,"prepare.run_wall_time_us":787,"replication_time_us":6524}]]}
I20260320 15:27:46.177450 28892 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_ce843e88d7f24053b0be457f201e11ae" } new_table_name: "default.Table_223074fa3ea142debceae9aae8ce1ce1" modify_external_catalogs: true
I20260320 15:27:46.179646 30466 raft_consensus.cc:493] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.179818 30466 raft_consensus.cc:515] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.180502 30466 leader_election.cc:290] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.181475 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
I20260320 15:27:46.181624 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet d823ab41194542fba5540e53e9f4680b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.181836 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.181159 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.182224 28440 raft_consensus.cc:2393] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:46.182900 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 288) took 4676 ms. Trace:
I20260320 15:27:46.182971 29643 rpcz_store.cc:276] 0320 15:27:41.506012 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.506064 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:46.182780 (+4676716us) ts_tablet_manager.cc:1151] Deleting tablet 87cb35cefee841d0881b08c456df5a97
0320 15:27:46.182792 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.182884 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4663605}
W20260320 15:27:46.183563 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 87cb35cefee841d0881b08c456df5a97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87cb35cefee841d0881b08c456df5a97 already in progress: creating tablet
I20260320 15:27:46.183813 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 291) took 3643 ms. Trace:
I20260320 15:27:46.172591 28904 rpcz_store.cc:276] 0320 15:27:43.925668 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.234370 (+1308702us) service_pool.cc:225] Handling call
0320 15:27:45.301749 (+ 67379us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.325006 (+ 23257us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.325214 (+ 208us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:45.325384 (+ 170us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:45.325457 (+ 73us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:45.482341 (+156884us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.482349 (+ 8us) write_op.cc:454] Released schema lock
0320 15:27:45.482456 (+ 107us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:46.172414 (+689958us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.473636 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.473792 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.473801 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:45.473804 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.473928 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.473945 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.473951 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:45.474016 (+ 65us) tablet.cc:700] Row locks acquired
0320 15:27:45.474019 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.474063 (+ 44us) write_op.cc:270] Start()
0320 15:27:45.474101 (+ 38us) write_op.cc:276] Timestamp: P: 1774020465474027 usec, L: 0
0320 15:27:45.474105 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.474367 (+ 262us) log.cc:844] Serialized 405 byte log entry
0320 15:27:45.478735 (+ 4368us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.478890 (+ 155us) write_op.cc:301] APPLY: starting
0320 15:27:45.478910 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.478956 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.478995 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.479054 (+ 59us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.479091 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.479098 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.479187 (+ 89us) log.cc:844] Serialized 23 byte log entry
0320 15:27:45.479253 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.479279 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.479287 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":99236608,"child_traces":[["op",{"apply.queue_time_us":95,"apply.run_cpu_time_us":453,"apply.run_wall_time_us":454,"num_ops":1,"prepare.queue_time_us":29805,"prepare.run_cpu_time_us":860,"prepare.run_wall_time_us":860,"raft.queue_time_us":535,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":7119,"replication_time_us":4591}]]}
I20260320 15:27:46.184775 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f832b79a92ff48718c372c5aee5ce446" } new_table_name: "default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd" modify_external_catalogs: true
I20260320 15:27:46.184492 29643 rpcz_store.cc:276] 0320 15:27:42.540429 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.540497 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:46.183704 (+3643207us) ts_tablet_manager.cc:1151] Deleting tablet aba79ddde6e842dfb67597c5d9e371cb
0320 15:27:46.183713 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.183798 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3642980}
W20260320 15:27:46.188649 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet aba79ddde6e842dfb67597c5d9e371cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba79ddde6e842dfb67597c5d9e371cb already in progress: creating tablet
I20260320 15:27:46.189788 29643 tablet_replica.cc:333] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.190071 29643 raft_consensus.cc:2243] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.190307 29643 raft_consensus.cc:2272] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.190908 29643 ts_tablet_manager.cc:1916] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.196187 30619 raft_consensus.cc:697] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.196569 30619 consensus_queue.cc:237] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.196978 28734 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.197115 28734 ts_tablet_manager.cc:1403] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20260320 15:27:46.197872 28734 raft_consensus.cc:359] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.198060 28734 raft_consensus.cc:385] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.198123 28734 raft_consensus.cc:740] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.198408 28734 consensus_queue.cc:260] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.199242 28734 ts_tablet_manager.cc:1434] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:46.199695 28734 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.199826 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet b16ffb28029e4b7ca37ce36efe9e0271
I20260320 15:27:46.200484 28889 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 134) took 2873 ms. Trace:
I20260320 15:27:46.201644 30355 raft_consensus.cc:493] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.201787 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.201849 30355 raft_consensus.cc:515] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:46.202041 28900 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 366df9b29cee4b2b9dfd72b0061f175a consensus state has no leader
W20260320 15:27:46.202834 28903 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 52ab0f33ad6b4cdfbe36e707872bee95 consensus state has no leader
I20260320 15:27:46.202836 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet b4cad900411e4c9686701aa99206dd23
I20260320 15:27:46.203495 29640 ts_tablet_manager.cc:1950] T 6771bd6a3f0e47d99a0191eae25cb2d2 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:46.203589 28901 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 137) took 2199 ms. Trace:
I20260320 15:27:46.203707 28901 rpcz_store.cc:276] 0320 15:27:44.004333 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.311234 (+1306901us) service_pool.cc:225] Handling call
0320 15:27:45.428541 (+117307us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.428950 (+ 409us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:45.429039 (+ 89us) catalog_manager.cc:2824] Locking tablets
0320 15:27:45.726193 (+297154us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:45.748644 (+ 22451us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:45.748744 (+ 100us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:46.202775 (+454031us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:46.203506 (+ 731us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.727523 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.727727 (+ 204us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.727737 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:45.727742 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:45.727878 (+ 136us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.727894 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.727899 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.728039 (+ 140us) tablet.cc:700] Row locks acquired
0320 15:27:45.728042 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.728086 (+ 44us) write_op.cc:270] Start()
0320 15:27:45.728125 (+ 39us) write_op.cc:276] Timestamp: P: 1774020465728027 usec, L: 0
0320 15:27:45.728130 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.728332 (+ 202us) log.cc:844] Serialized 765 byte log entry
0320 15:27:45.747793 (+ 19461us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.747929 (+ 136us) write_op.cc:301] APPLY: starting
0320 15:27:45.747952 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.748007 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.748046 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.748131 (+ 85us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.748172 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.748182 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:45.748275 (+ 93us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.748360 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.748387 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:45.748395 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.748488 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.748505 (+ 17us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":73,"apply.run_cpu_time_us":547,"apply.run_wall_time_us":547,"num_ops":2,"prepare.queue_time_us":76,"prepare.run_cpu_time_us":903,"prepare.run_wall_time_us":903,"replication_time_us":19622}]]}
I20260320 15:27:46.204716 28901 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_44d6e0bede5f4c87971a0e127aef1be0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:46.205430 28901 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44d6e0bede5f4c87971a0e127aef1be0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.205745 28734 tablet_bootstrap.cc:654] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.206638 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet b4cad900411e4c9686701aa99206dd23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.208487 30355 leader_election.cc:290] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.210347 28736 tablet_bootstrap.cc:492] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
W20260320 15:27:46.210513 28522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260320 15:27:46.210700 28736 ts_tablet_manager.cc:1403] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.053s user 0.004s sys 0.000s
I20260320 15:27:46.211779 29939 leader_election.cc:290] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.212769 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 177) took 10458 ms. Trace:
I20260320 15:27:46.212374 28736 raft_consensus.cc:359] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.212918 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.212932 28736 raft_consensus.cc:385] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.213222 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet d823ab41194542fba5540e53e9f4680b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.213366 28736 raft_consensus.cc:740] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.220680 28733 tablet_bootstrap.cc:654] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.213687 29643 ts_tablet_manager.cc:1929] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:46.216094 28736 consensus_queue.cc:260] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.223306 29643 log.cc:1199] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:46.223769 29643 ts_tablet_manager.cc:1950] T 2e9791ac61b34fc09a8292c79f0c4032 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.225193 30395 raft_consensus.cc:515] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.225952 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 292) took 3101 ms. Trace:
I20260320 15:27:46.226048 29643 rpcz_store.cc:276] 0320 15:27:43.124624 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.124685 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:46.189729 (+3065044us) ts_tablet_manager.cc:1151] Deleting tablet 2e9791ac61b34fc09a8292c79f0c4032
0320 15:27:46.189741 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.195435 (+ 5694us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.213540 (+ 18105us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.225839 (+ 12299us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3064832}
I20260320 15:27:46.226603 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 2e9791ac61b34fc09a8292c79f0c4032 (table default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]) successfully deleted
I20260320 15:27:46.226477 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 304) took 2486 ms. Trace:
I20260320 15:27:46.230927 29643 rpcz_store.cc:276] 0320 15:27:43.739746 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.739817 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:46.226342 (+2486525us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:46.226358 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.226458 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2486281}
W20260320 15:27:46.231717 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because the tablet was not found. No further retry: Not found: Tablet not found: 0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:46.232932 29643 tablet_replica.cc:333] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.233196 29643 raft_consensus.cc:2243] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.233415 29643 raft_consensus.cc:2272] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.233631 28736 ts_tablet_manager.cc:1434] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.023s user 0.000s sys 0.003s
I20260320 15:27:46.235139 29643 ts_tablet_manager.cc:1916] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.240942 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.244314 28736 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.248136 28736 tablet_bootstrap.cc:654] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.258389 29643 ts_tablet_manager.cc:1929] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.259408 29643 log.cc:1199] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/41da3cb06b1147fbae2695bc5c9ffaa5
I20260320 15:27:46.260267 29643 ts_tablet_manager.cc:1950] T 41da3cb06b1147fbae2695bc5c9ffaa5 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.262208 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 305) took 2522 ms. Trace:
I20260320 15:27:46.264456 29643 rpcz_store.cc:276] 0320 15:27:43.739870 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740003 (+ 133us) service_pool.cc:225] Handling call
0320 15:27:46.232846 (+2492843us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:46.232863 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.244666 (+ 11803us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.258221 (+ 13555us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.262180 (+ 3959us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2492642}
I20260320 15:27:46.265628 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
I20260320 15:27:46.266273 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 306) took 2526 ms. Trace:
I20260320 15:27:46.266266 30395 leader_election.cc:290] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.267174 29643 rpcz_store.cc:276] 0320 15:27:43.740007 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740233 (+ 226us) service_pool.cc:225] Handling call
0320 15:27:46.266143 (+2525910us) ts_tablet_manager.cc:1151] Deleting tablet fa5618796f1a4ff3ab4c1209dad0aede
0320 15:27:46.266157 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.266253 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2525772}
W20260320 15:27:46.268330 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet fa5618796f1a4ff3ab4c1209dad0aede because the tablet was not found. No further retry: Not found: Tablet not found: fa5618796f1a4ff3ab4c1209dad0aede
I20260320 15:27:46.268934 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 309) took 2528 ms. Trace:
I20260320 15:27:46.270419 28552 tablet_service.cc:1558] Processing DeleteTablet for tablet b4cad900411e4c9686701aa99206dd23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.271073 29643 rpcz_store.cc:276] 0320 15:27:43.740322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740396 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:46.268812 (+2528416us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:46.268827 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.268914 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2528291}
W20260320 15:27:46.272220 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 1bf5e17dea5445d9909a229fa8315348 because the tablet was not found. No further retry: Not found: Tablet not found: 1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:46.272778 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 310) took 2532 ms. Trace:
I20260320 15:27:46.273499 29643 rpcz_store.cc:276] 0320 15:27:43.740390 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740539 (+ 149us) service_pool.cc:225] Handling call
0320 15:27:46.272667 (+2532128us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:46.272679 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.272760 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2531981}
W20260320 15:27:46.271174 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 85839.9us
I20260320 15:27:46.274969 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:46.275125 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 3e7b964395f24e45bc2715be3fac6dbe because the tablet was not found. No further retry: Not found: Tablet not found: 3e7b964395f24e45bc2715be3fac6dbe
I20260320 15:27:46.275218 28444 raft_consensus.cc:3060] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.275434 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.275640 28440 raft_consensus.cc:2468] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.275892 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.276031 28446 raft_consensus.cc:3060] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.278038 28444 raft_consensus.cc:2468] T c3a49e310b524f3b87eb89f3e84b7800 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.278477 28446 raft_consensus.cc:2468] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.280170 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.280387 28302 raft_consensus.cc:2468] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.280903 28300 raft_consensus.cc:3060] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.281629 28304 raft_consensus.cc:3060] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.200654 28889 rpcz_store.cc:276] 0320 15:27:43.327275 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.234323 (+1907048us) service_pool.cc:225] Handling call
0320 15:27:45.256304 (+ 21981us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf7bf6 00007fe6118e7486 00007fe6118e71b5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.326073 (+ 69769us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.326220 (+ 147us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:45.326270 (+ 50us) catalog_manager.cc:2824] Locking tablets
0320 15:27:45.326430 (+ 160us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:45.349892 (+ 23462us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:45.353796 (+ 3904us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:46.199763 (+845967us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:46.200401 (+ 638us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.340252 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.340413 (+ 161us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.340421 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:45.340425 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:45.340563 (+ 138us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.340580 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.340587 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.340708 (+ 121us) tablet.cc:700] Row locks acquired
0320 15:27:45.340710 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:45.340752 (+ 42us) write_op.cc:270] Start()
0320 15:27:45.340794 (+ 42us) write_op.cc:276] Timestamp: P: 1774020465340727 usec, L: 0
0320 15:27:45.340797 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.340978 (+ 181us) log.cc:844] Serialized 725 byte log entry
0320 15:27:45.349311 (+ 8333us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.349458 (+ 147us) write_op.cc:301] APPLY: starting
0320 15:27:45.349474 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.349520 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.349554 (+ 34us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.349619 (+ 65us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.349649 (+ 30us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.349656 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.349727 (+ 71us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.349775 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.349801 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.349808 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":49733888,"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":430,"apply.run_wall_time_us":429,"num_ops":2,"prepare.queue_time_us":13118,"prepare.run_cpu_time_us":832,"prepare.run_wall_time_us":832,"raft.queue_time_us":1009,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":8493}]]}
I20260320 15:27:46.284646 28524 leader_election.cc:304] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.284992 28524 leader_election.cc:304] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.284940 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_180c8e69093147538db5cd1b8489fa6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:46.285218 28524 leader_election.cc:304] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.285678 30642 raft_consensus.cc:2804] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.285755 30642 raft_consensus.cc:493] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260320 15:27:46.285753 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_180c8e69093147538db5cd1b8489fa6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.285826 30642 raft_consensus.cc:3060] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.286034 29939 raft_consensus.cc:2804] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.287551 30629 raft_consensus.cc:2804] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.288101 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 311) took 2547 ms. Trace:
I20260320 15:27:46.289054 29643 rpcz_store.cc:276] 0320 15:27:43.740517 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740725 (+ 208us) service_pool.cc:225] Handling call
0320 15:27:46.287974 (+2547249us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:46.287987 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.288076 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2547118}
W20260320 15:27:46.290442 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because the tablet was not found. No further retry: Not found: Tablet not found: b6201b74fdb0428e98f2eebb0b24d01a
I20260320 15:27:46.291143 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 308) took 2550 ms. Trace:
I20260320 15:27:46.292004 29643 rpcz_store.cc:276] 0320 15:27:43.740239 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740882 (+ 643us) service_pool.cc:225] Handling call
0320 15:27:46.291013 (+2550131us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:46.291031 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.291124 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2550006}
W20260320 15:27:46.293179 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because the tablet was not found. No further retry: Not found: Tablet not found: ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:46.294075 28733 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:46.294410 28733 ts_tablet_manager.cc:1403] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.371s user 0.017s sys 0.007s
I20260320 15:27:46.294497 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 307) took 2554 ms. Trace:
I20260320 15:27:46.295423 29643 rpcz_store.cc:276] 0320 15:27:43.740136 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.740955 (+ 819us) service_pool.cc:225] Handling call
0320 15:27:46.293717 (+2552762us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:46.293735 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.294467 (+ 732us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2552639}
W20260320 15:27:46.296835 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 5fa5f6fe801b4452bf294a9ae23bb163 because the tablet was not found. No further retry: Not found: Tablet not found: 5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:46.297473 29643 tablet_replica.cc:333] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.297191 28733 raft_consensus.cc:359] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.297854 28733 raft_consensus.cc:385] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.298067 28733 raft_consensus.cc:740] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.298478 29643 raft_consensus.cc:2243] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:46.298816 28733 consensus_queue.cc:260] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.299933 29643 raft_consensus.cc:2272] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.301147 29643 ts_tablet_manager.cc:1916] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.302754 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85c397eab0ed4df188b0dc2a58077b39" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.302975 28300 raft_consensus.cc:3060] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.306262 28300 raft_consensus.cc:2468] T 85c397eab0ed4df188b0dc2a58077b39 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:46.307343 28873 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 52ab0f33ad6b4cdfbe36e707872bee95 consensus state has no leader
I20260320 15:27:46.312999 30642 raft_consensus.cc:515] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.314050 30642 leader_election.cc:290] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.316394 28296 raft_consensus.cc:3060] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.324095 30456 raft_consensus.cc:493] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.324333 30456 raft_consensus.cc:515] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.325349 30456 leader_election.cc:290] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.328860 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:46.330236 30395 consensus_queue.cc:1048] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.331900 30466 raft_consensus.cc:493] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.332254 30466 raft_consensus.cc:515] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.332439 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197f1935099140a98c5eb801cf44046a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.333062 28382 leader_election.cc:343] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.333437 30466 leader_election.cc:290] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.333745 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.333982 28443 raft_consensus.cc:2393] T 321fe367ec3348f8a049167f3f8c15d5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:46.337389 29939 raft_consensus.cc:697] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.314577 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.338119 28302 raft_consensus.cc:3060] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.338003 29939 consensus_queue.cc:237] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.340938 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.341166 28441 raft_consensus.cc:3060] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.341142 28302 raft_consensus.cc:2468] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.343189 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.342293 28294 raft_consensus.cc:3060] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.346024 28441 raft_consensus.cc:2468] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.331777 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.347390 28298 raft_consensus.cc:2468] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.352155 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 136) took 2426 ms. Trace:
I20260320 15:27:46.352394 28903 rpcz_store.cc:276] 0320 15:27:43.925774 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.294287 (+1368513us) service_pool.cc:225] Handling call
0320 15:27:45.323047 (+ 28760us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.323610 (+ 563us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:45.327416 (+ 3806us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:45.327477 (+ 61us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:45.351372 (+ 23895us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:46.352025 (+1000653us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.341135 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.341235 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.341240 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:45.341242 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.341330 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.341341 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.341346 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:45.341393 (+ 47us) tablet.cc:700] Row locks acquired
0320 15:27:45.341396 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:45.341425 (+ 29us) write_op.cc:270] Start()
0320 15:27:45.341454 (+ 29us) write_op.cc:276] Timestamp: P: 1774020465341403 usec, L: 0
0320 15:27:45.341458 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.341592 (+ 134us) log.cc:844] Serialized 405 byte log entry
0320 15:27:45.350628 (+ 9036us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.350725 (+ 97us) write_op.cc:301] APPLY: starting
0320 15:27:45.350746 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.350793 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.350826 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.350885 (+ 59us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.350929 (+ 44us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.350941 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:45.351032 (+ 91us) log.cc:844] Serialized 23 byte log entry
0320 15:27:45.351085 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.351103 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:45.351108 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":45,"apply.run_cpu_time_us":437,"apply.run_wall_time_us":438,"num_ops":1,"prepare.queue_time_us":5237,"prepare.run_cpu_time_us":536,"prepare.run_wall_time_us":536,"raft.queue_time_us":441,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":9111}]]}
I20260320 15:27:46.353983 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_228877df377e4f8bbe522f4d57092b2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:46.354830 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_228877df377e4f8bbe522f4d57092b2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.353384 28524 leader_election.cc:304] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:46.353470 28892 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26ae379304674bc3941af125f0d9d6b3 consensus state has no leader
I20260320 15:27:46.355985 30655 raft_consensus.cc:2804] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.356007 28892 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 143) took 1375 ms. Trace:
I20260320 15:27:46.356277 30655 raft_consensus.cc:697] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.356129 28892 rpcz_store.cc:276] 0320 15:27:44.980907 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.177150 (+1196243us) service_pool.cc:225] Handling call
0320 15:27:46.177693 (+ 543us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.177987 (+ 294us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:46.178282 (+ 295us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:46.178405 (+ 123us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:46.195628 (+ 17223us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:46.355936 (+160308us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.179173 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.179326 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.179337 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:46.179342 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:46.179467 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.179484 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.179488 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:46.179599 (+ 111us) tablet.cc:700] Row locks acquired
0320 15:27:46.179602 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.179647 (+ 45us) write_op.cc:270] Start()
0320 15:27:46.179685 (+ 38us) write_op.cc:276] Timestamp: P: 1774020466179589 usec, L: 0
0320 15:27:46.179688 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.179902 (+ 214us) log.cc:844] Serialized 405 byte log entry
0320 15:27:46.185608 (+ 5706us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.185736 (+ 128us) write_op.cc:301] APPLY: starting
0320 15:27:46.185757 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.185801 (+ 44us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.185845 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.185898 (+ 53us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.185930 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:46.185937 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:46.186024 (+ 87us) log.cc:844] Serialized 23 byte log entry
0320 15:27:46.186099 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.186127 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:46.186136 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.186225 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.186229 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":480,"num_ops":1,"prepare.queue_time_us":68,"prepare.run_cpu_time_us":846,"prepare.run_wall_time_us":847,"raft.queue_time_us":1392,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":322,"replication_time_us":5888}]]}
I20260320 15:27:46.356585 30655 consensus_queue.cc:237] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.352192 28900 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 133) took 3031 ms. Trace:
I20260320 15:27:46.356963 28900 rpcz_store.cc:276] 0320 15:27:43.320358 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.233184 (+1912826us) service_pool.cc:225] Handling call
0320 15:27:45.253251 (+ 20067us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:45.322937 (+ 69686us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:45.324158 (+ 1221us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:45.324375 (+ 217us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:45.324457 (+ 82us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:45.354655 (+ 30198us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:46.352082 (+997427us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.325061 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.325180 (+ 119us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.325186 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:45.325188 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:45.325325 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.325341 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.325346 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:45.325415 (+ 69us) tablet.cc:700] Row locks acquired
0320 15:27:45.325417 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:45.325457 (+ 40us) write_op.cc:270] Start()
0320 15:27:45.325495 (+ 38us) write_op.cc:276] Timestamp: P: 1774020465325435 usec, L: 0
0320 15:27:45.325501 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.339435 (+ 13934us) log.cc:844] Serialized 405 byte log entry
0320 15:27:45.340127 (+ 692us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:45.349212 (+ 9085us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.349400 (+ 188us) write_op.cc:301] APPLY: starting
0320 15:27:45.349425 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.349479 (+ 54us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.349484 (+ 5us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.349563 (+ 79us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.349603 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.349613 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:45.349714 (+ 101us) log.cc:844] Serialized 23 byte log entry
0320 15:27:45.349816 (+ 102us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.349838 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:45.349847 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.349935 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.349941 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":45224320,"child_traces":[["op",{"apply.queue_time_us":119,"apply.run_cpu_time_us":520,"apply.run_wall_time_us":520,"num_ops":1,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":1201,"prepare.run_wall_time_us":15154,"raft.queue_time_us":1550,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":303,"replication_time_us":23665,"spinlock_wait_cycles":31402752}]]}
I20260320 15:27:46.358068 28900 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_0c66e258345e4633aea4d4962f922159" } modify_external_catalogs: true
I20260320 15:27:46.358047 28424 tablet_service.cc:1511] Processing CreateTablet for tablet 9d7f7b99899c4001adcbdce58910e717 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.359586 28900 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_0c66e258345e4633aea4d4962f922159" } modify_external_catalogs: true
I20260320 15:27:46.358526 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_4478d6addfd34e128e575a94d94c09f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:46.359884 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 806d635bb5594f0bb826b9b31468b485 (DEFAULT_TABLE table=default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:46.360347 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4478d6addfd34e128e575a94d94c09f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.360702 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2003e2af9a4d9c854990e1db41af28 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.378542 28900 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet db16d86664794305bde714a7ff4d5caf
I20260320 15:27:46.378911 28736 tablet_bootstrap.cc:492] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.379063 28736 ts_tablet_manager.cc:1403] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.140s user 0.004s sys 0.004s
I20260320 15:27:46.380071 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 806d635bb5594f0bb826b9b31468b485 (DEFAULT_TABLE table=default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.380697 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d635bb5594f0bb826b9b31468b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.380710 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet db16d86664794305bde714a7ff4d5caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.381175 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet db16d86664794305bde714a7ff4d5caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.383380 30655 consensus_queue.cc:1048] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.389106 28900 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 148) took 1089 ms. Trace:
I20260320 15:27:46.389904 28900 rpcz_store.cc:276] 0320 15:27:45.299311 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.357779 (+1058468us) service_pool.cc:225] Handling call
0320 15:27:46.359791 (+ 2012us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.360575 (+ 784us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:46.360635 (+ 60us) catalog_manager.cc:2824] Locking tablets
0320 15:27:46.360778 (+ 143us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:46.378270 (+ 17492us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:46.378369 (+ 99us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:46.378437 (+ 68us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:46.389049 (+ 10612us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.364196 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.364316 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.364322 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:46.364325 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.364453 (+ 128us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.364466 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.364472 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.364583 (+ 111us) tablet.cc:700] Row locks acquired
0320 15:27:46.364586 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.364621 (+ 35us) write_op.cc:270] Start()
0320 15:27:46.364655 (+ 34us) write_op.cc:276] Timestamp: P: 1774020466364597 usec, L: 0
0320 15:27:46.364658 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.364846 (+ 188us) log.cc:844] Serialized 756 byte log entry
0320 15:27:46.377239 (+ 12393us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.377704 (+ 465us) write_op.cc:301] APPLY: starting
0320 15:27:46.377724 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.377777 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.377817 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.377898 (+ 81us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.377938 (+ 40us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:46.377951 (+ 13us) write_op.cc:312] APPLY: finished
0320 15:27:46.378063 (+ 112us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.378131 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.378157 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:46.378163 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.378242 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.378245 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":391,"apply.run_cpu_time_us":536,"apply.run_wall_time_us":536,"num_ops":2,"prepare.queue_time_us":865,"prepare.run_cpu_time_us":739,"prepare.run_wall_time_us":739,"replication_time_us":12542}]]}
I20260320 15:27:46.393692 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7f7b99899c4001adcbdce58910e717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.393834 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 9d7f7b99899c4001adcbdce58910e717 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.394829 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7f7b99899c4001adcbdce58910e717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.394790 28736 raft_consensus.cc:359] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.394966 28736 raft_consensus.cc:385] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.395015 28736 raft_consensus.cc:740] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.395221 28736 consensus_queue.cc:260] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.399210 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d635bb5594f0bb826b9b31468b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.401108 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2003e2af9a4d9c854990e1db41af28 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.401584 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2003e2af9a4d9c854990e1db41af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.392745 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.403312 28297 raft_consensus.cc:2468] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.403888 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2003e2af9a4d9c854990e1db41af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.404322 29678 ts_tablet_manager.cc:1929] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.404451 29678 log.cc:1199] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:46.404374 28551 tablet_service.cc:1511] Processing CreateTablet for tablet f77a32615c954da581e9a9e4cb45c497 (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.404773 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f77a32615c954da581e9a9e4cb45c497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.405285 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.405458 28304 raft_consensus.cc:2468] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.406471 29678 ts_tablet_manager.cc:1950] T 06cb44b9cc6445cf8324cf52190d6773 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:46.407176 28304 raft_consensus.cc:3060] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.410847 28903 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 147) took 1116 ms. Trace:
I20260320 15:27:46.410701 28413 tablet_service.cc:1511] Processing CreateTablet for tablet f77a32615c954da581e9a9e4cb45c497 (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.411679 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f77a32615c954da581e9a9e4cb45c497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.411736 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.412000 28304 raft_consensus.cc:3060] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.412617 28903 rpcz_store.cc:276] 0320 15:27:45.294082 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.353703 (+1059621us) service_pool.cc:225] Handling call
0320 15:27:46.355130 (+ 1427us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.358901 (+ 3771us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.410462 (+ 51561us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.410622 (+ 160us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.410760 (+ 138us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.360934 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.361026 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.361032 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:46.361035 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.361125 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.361138 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.361143 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.361214 (+ 71us) tablet.cc:700] Row locks acquired
0320 15:27:46.361216 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:46.361246 (+ 30us) write_op.cc:270] Start()
0320 15:27:46.361281 (+ 35us) write_op.cc:276] Timestamp: P: 1774020466361224 usec, L: 0
0320 15:27:46.361285 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.361432 (+ 147us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.368960 (+ 7528us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.370314 (+ 1354us) write_op.cc:301] APPLY: starting
0320 15:27:46.370339 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.370397 (+ 58us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.370432 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.370598 (+ 166us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.370641 (+ 43us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.370649 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:46.370755 (+ 106us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.370872 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.370903 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:46.370936 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.371052 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.371058 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1252,"apply.run_cpu_time_us":734,"apply.run_wall_time_us":734,"num_ops":2,"prepare.queue_time_us":1452,"prepare.run_cpu_time_us":559,"prepare.run_wall_time_us":558,"replication_time_us":7633}]]}
I20260320 15:27:46.413118 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.414178 30466 raft_consensus.cc:493] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.414324 30466 raft_consensus.cc:515] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.414572 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_74ed94505ea049559e4e388efe568e9b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:46.414906 30466 leader_election.cc:290] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.415436 28736 ts_tablet_manager.cc:1434] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.036s user 0.002s sys 0.000s
I20260320 15:27:46.415829 28736 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.415934 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_fe5422b8425b4fe9988570705b9fddcd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:46.416558 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe5422b8425b4fe9988570705b9fddcd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.417352 28736 tablet_bootstrap.cc:654] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.418489 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.419629 28889 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 146) took 1389 ms. Trace:
I20260320 15:27:46.420195 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.420445 28446 raft_consensus.cc:2468] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
W20260320 15:27:46.420722 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74ed94505ea049559e4e388efe568e9b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.422102 28240 leader_election.cc:304] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.422542 30466 raft_consensus.cc:2804] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.422628 30466 raft_consensus.cc:493] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.422693 30466 raft_consensus.cc:3060] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.423600 28903 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_3aeb90fb3c79479ea771ab8d88f6bbf4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:46.424201 28903 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3aeb90fb3c79479ea771ab8d88f6bbf4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.425648 30466 raft_consensus.cc:515] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.427116 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.427392 28446 raft_consensus.cc:3060] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.427465 28892 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_8af7bea7b0024b3c93158cfcb8d04bbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:46.428128 28892 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8af7bea7b0024b3c93158cfcb8d04bbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.429864 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 275) took 2620 ms. Trace:
I20260320 15:27:46.429971 29678 rpcz_store.cc:276] 0320 15:27:43.808981 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.809073 (+ 92us) service_pool.cc:225] Handling call
0320 15:27:46.015490 (+2206417us) ts_tablet_manager.cc:1151] Deleting tablet 06cb44b9cc6445cf8324cf52190d6773
0320 15:27:46.015656 (+ 166us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.135404 (+119748us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.387451 (+252047us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.429836 (+ 42385us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22464,"tablet-delete.queue_time_us":2205552}
I20260320 15:27:46.430290 28446 raft_consensus.cc:2468] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:46.430356 29678 tablet_replica.cc:333] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:46.430562 29678 raft_consensus.cc:2243] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.430725 29678 raft_consensus.cc:2272] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.431322 29678 ts_tablet_manager.cc:1916] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.431857 28240 leader_election.cc:304] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.432231 30623 raft_consensus.cc:2804] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.432380 30623 raft_consensus.cc:697] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:46.432704 30623 consensus_queue.cc:237] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.439827 30456 raft_consensus.cc:493] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.440018 30456 raft_consensus.cc:515] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.440891 30456 leader_election.cc:290] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.441144 28901 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 145) took 1426 ms. Trace:
I20260320 15:27:46.442130 30466 leader_election.cc:290] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:46.441915 28901 rpcz_store.cc:276] 0320 15:27:45.014286 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.204454 (+1190168us) service_pool.cc:225] Handling call
0320 15:27:46.399588 (+195134us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.411638 (+ 12050us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.440047 (+ 28409us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.440903 (+ 856us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.441051 (+ 148us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.412464 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.412635 (+ 171us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.412644 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:46.412647 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.412761 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.412789 (+ 28us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.412795 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.412900 (+ 105us) tablet.cc:700] Row locks acquired
0320 15:27:46.412903 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.412958 (+ 55us) write_op.cc:270] Start()
0320 15:27:46.413004 (+ 46us) write_op.cc:276] Timestamp: P: 1774020466412922 usec, L: 0
0320 15:27:46.413008 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.418599 (+ 5591us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.418876 (+ 277us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612f63f22 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f 00007fe6070d827b 00007fe6070d80f8 0000000000a8b58c
0320 15:27:46.425504 (+ 6628us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.425709 (+ 205us) write_op.cc:301] APPLY: starting
0320 15:27:46.425729 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.425781 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.425784 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.425910 (+ 126us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.425947 (+ 37us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.425953 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:46.426040 (+ 87us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.426121 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.426143 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:46.426149 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.426219 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.426223 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":148,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":500,"num_ops":2,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":1485,"prepare.run_wall_time_us":6513,"raft.queue_time_us":1104,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":315,"replication_time_us":12456,"spinlock_wait_cycles":11839232}]]}
I20260320 15:27:46.444031 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.444619 28382 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.420197 28889 rpcz_store.cc:276] 0320 15:27:45.030454 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.284597 (+1254143us) service_pool.cc:225] Handling call
0320 15:27:46.351946 (+ 67349us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.352712 (+ 766us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.419137 (+ 66425us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.419144 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:46.419231 (+ 87us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.419409 (+ 178us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.419517 (+ 108us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.353760 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.353948 (+ 188us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.353956 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:46.353960 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:46.354085 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.354115 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.354127 (+ 12us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.354219 (+ 92us) tablet.cc:700] Row locks acquired
0320 15:27:46.354222 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.354283 (+ 61us) write_op.cc:270] Start()
0320 15:27:46.354326 (+ 43us) write_op.cc:276] Timestamp: P: 1774020466354243 usec, L: 0
0320 15:27:46.354330 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.354577 (+ 247us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.360741 (+ 6164us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.360898 (+ 157us) write_op.cc:301] APPLY: starting
0320 15:27:46.360923 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.361019 (+ 96us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.361063 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.361205 (+ 142us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.361255 (+ 50us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.361263 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:46.361372 (+ 109us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.361456 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.361481 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:46.361490 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":676,"apply.run_wall_time_us":675,"num_ops":2,"prepare.queue_time_us":281,"prepare.run_cpu_time_us":943,"prepare.run_wall_time_us":943,"raft.queue_time_us":100,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":278,"replication_time_us":6369}]]}
I20260320 15:27:46.449998 30629 raft_consensus.cc:697] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.451318 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_b74d0dcbf6da41dfbf9b6bad57798e6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:46.451834 30629 consensus_queue.cc:237] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.459019 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.460142 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "325caab9b8f34f04a97b41137652ccdb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.460355 28296 raft_consensus.cc:3060] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.463359 28296 raft_consensus.cc:2468] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.467597 28296 raft_consensus.cc:3060] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.471817 30456 raft_consensus.cc:493] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.472035 30456 raft_consensus.cc:515] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.465790 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 151) took 1030 ms. Trace:
I20260320 15:27:46.472981 30456 leader_election.cc:290] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.473068 28900 rpcz_store.cc:276] 0320 15:27:45.434830 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.391099 (+956269us) service_pool.cc:225] Handling call
0320 15:27:46.414486 (+ 23387us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:46.420879 (+ 6393us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.421422 (+ 543us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.465471 (+ 44049us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.465616 (+ 145us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.465731 (+ 115us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.422032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.422154 (+ 122us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.422159 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:46.422162 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.422250 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.422261 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.422265 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.422339 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:46.422341 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:46.422378 (+ 37us) write_op.cc:270] Start()
0320 15:27:46.422416 (+ 38us) write_op.cc:276] Timestamp: P: 1774020466422347 usec, L: 0
0320 15:27:46.422420 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.422678 (+ 258us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.459555 (+ 36877us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.459698 (+ 143us) write_op.cc:301] APPLY: starting
0320 15:27:46.459766 (+ 68us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.459825 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.459867 (+ 42us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.463711 (+ 3844us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.463757 (+ 46us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.463765 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:46.465248 (+ 1483us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.465336 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.465362 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:46.465370 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.465949 (+ 579us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.465955 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":52816256,"child_traces":[["op",{"apply.queue_time_us":32,"apply.run_cpu_time_us":784,"apply.run_wall_time_us":6238,"num_ops":2,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":741,"prepare.run_wall_time_us":740,"replication_time_us":37061}]]}
I20260320 15:27:46.453303 30619 raft_consensus.cc:493] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.475713 30619 raft_consensus.cc:515] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.476476 30619 leader_election.cc:290] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.477560 29643 ts_tablet_manager.cc:1929] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.477718 29643 log.cc:1199] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:46.477947 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20eec704800e4d8b924d97556cc2700e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.478621 28382 leader_election.cc:343] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:46.481909 28524 consensus_peers.cc:597] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cea3bf14653f4555acfed1577a79ee4f. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.456074 30630 raft_consensus.cc:493] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.482314 30630 raft_consensus.cc:515] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:46.482539 28524 consensus_peers.cc:597] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a4b5a658cd84b04b50b54973c065889. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.483151 30630 leader_election.cc:290] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:46.483274 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b74d0dcbf6da41dfbf9b6bad57798e6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.466560 28903 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 152) took 1030 ms. Trace:
I20260320 15:27:46.484411 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.450719 28901 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_5ef08b954dbd41f7b7ff75d7d07515dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:46.484614 28441 raft_consensus.cc:2468] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.484241 28903 rpcz_store.cc:276] 0320 15:27:45.436125 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.413339 (+977214us) service_pool.cc:225] Handling call
0320 15:27:46.423472 (+ 10133us) spinlock_profiling.cc:243] Waited 9.78 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:46.424300 (+ 828us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.425227 (+ 927us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.466191 (+ 40964us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.466325 (+ 134us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.466462 (+ 137us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.426050 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.426190 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.426195 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:46.426197 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:46.426286 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.426298 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.426302 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.426374 (+ 72us) tablet.cc:700] Row locks acquired
0320 15:27:46.426377 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.426420 (+ 43us) write_op.cc:270] Start()
0320 15:27:46.426453 (+ 33us) write_op.cc:276] Timestamp: P: 1774020466426389 usec, L: 0
0320 15:27:46.426457 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.426659 (+ 202us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.465041 (+ 38382us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.465676 (+ 635us) write_op.cc:301] APPLY: starting
0320 15:27:46.465695 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.465735 (+ 40us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.465809 (+ 74us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.465910 (+ 101us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.465943 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.465950 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:46.466021 (+ 71us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.466083 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.466099 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:46.466105 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.466200 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.466204 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":22496512,"child_traces":[["op",{"apply.queue_time_us":553,"apply.run_cpu_time_us":508,"apply.run_wall_time_us":507,"num_ops":2,"prepare.queue_time_us":186,"prepare.run_cpu_time_us":675,"prepare.run_wall_time_us":673,"replication_time_us":37929}]]}
I20260320 15:27:46.485057 28903 master_service.cc:954] ReplaceTablet: received request to replace tablet 49654c67435a48448794879464f6cfb2 from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:46.485180 28901 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ef08b954dbd41f7b7ff75d7d07515dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:46.487020 28441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.487291 28441 raft_consensus.cc:2468] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.487974 28524 leader_election.cc:304] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.458820 30651 raft_consensus.cc:493] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.464062 30394 raft_consensus.cc:493] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.465386 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:46.471697 28733 ts_tablet_manager.cc:1434] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.177s user 0.003s sys 0.000s
I20260320 15:27:46.488306 30394 raft_consensus.cc:515] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.488423 28892 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 153) took 1052 ms. Trace:
I20260320 15:27:46.488555 28733 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:46.488526 28892 rpcz_store.cc:276] 0320 15:27:45.436246 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.427185 (+990939us) service_pool.cc:225] Handling call
0320 15:27:46.428238 (+ 1053us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.428785 (+ 547us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.488089 (+ 59304us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.488228 (+ 139us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.488350 (+ 122us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.430118 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.430243 (+ 125us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.430250 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:46.430252 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:46.430388 (+ 136us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.430402 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.430406 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.430502 (+ 96us) tablet.cc:700] Row locks acquired
0320 15:27:46.430505 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.430546 (+ 41us) write_op.cc:270] Start()
0320 15:27:46.430587 (+ 41us) write_op.cc:276] Timestamp: P: 1774020466430515 usec, L: 0
0320 15:27:46.430591 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.430795 (+ 204us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.465182 (+ 34387us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.466046 (+ 864us) write_op.cc:301] APPLY: starting
0320 15:27:46.466062 (+ 16us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.466102 (+ 40us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.466708 (+ 606us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.466849 (+ 141us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.466891 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.466902 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:46.466986 (+ 84us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.467057 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.467081 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:46.467090 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":817,"num_ops":2,"prepare.queue_time_us":611,"prepare.run_cpu_time_us":758,"prepare.run_wall_time_us":759,"replication_time_us":34552,"threads_started":1}]]}
I20260320 15:27:46.488942 30394 leader_election.cc:290] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.489207 28892 master_service.cc:954] ReplaceTablet: received request to replace tablet 7840b529206d482283cc550d2c5e7606 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:46.490417 28733 tablet_bootstrap.cc:654] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.493942 30651 raft_consensus.cc:515] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.498950 30651 leader_election.cc:290] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.504632 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.505743 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3a49e310b524f3b87eb89f3e84b7800" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.505930 28294 raft_consensus.cc:3060] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.508416 28551 tablet_service.cc:1511] Processing CreateTablet for tablet a8b40dfe348b48ea8d941e3d97f27d73 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.508880 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b40dfe348b48ea8d941e3d97f27d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.509660 29679 ts_manager.cc:194] Registered new tserver with Master: 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:46.513001 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.513046 28413 tablet_service.cc:1511] Processing CreateTablet for tablet a8b40dfe348b48ea8d941e3d97f27d73 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.513237 28303 raft_consensus.cc:2468] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.513885 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.514050 28303 raft_consensus.cc:2468] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.514109 29643 ts_tablet_manager.cc:1950] T 06cb44b9cc6445cf8324cf52190d6773 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.514477 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet cea3bf14653f4555acfed1577a79ee4f (table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]) successfully deleted
I20260320 15:27:46.514806 30394 raft_consensus.cc:2804] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260320 15:27:46.514853 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ea95b7a1b30e42cebcbfcd551c58f679 because the tablet was not found. No further retry: Not found: Tablet not found: ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:46.514887 30394 raft_consensus.cc:493] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.514963 30394 raft_consensus.cc:3060] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:46.515125 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because the tablet was not found. No further retry: Not found: Tablet not found: 66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:46.515458 28524 leader_election.cc:304] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:46.515610 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 48c1002cbdd845e2b96dcb57c450cc6d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48c1002cbdd845e2b96dcb57c450cc6d already in progress: creating tablet
W20260320 15:27:46.515859 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 26818e620ff649d18b671e4f4c7c7e74 because the tablet was not found. No further retry: Not found: Tablet not found: 26818e620ff649d18b671e4f4c7c7e74
W20260320 15:27:46.516170 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because the tablet was not found. No further retry: Not found: Tablet not found: 7c4b53fb135741efad5d3e59a6d4eb3e
W20260320 15:27:46.516415 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3e7b964395f24e45bc2715be3fac6dbe because the tablet was not found. No further retry: Not found: Tablet not found: 3e7b964395f24e45bc2715be3fac6dbe
W20260320 15:27:46.516845 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1bf5e17dea5445d9909a229fa8315348 because the tablet was not found. No further retry: Not found: Tablet not found: 1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:46.517884 30394 raft_consensus.cc:515] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.518154 29939 raft_consensus.cc:2804] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.518551 29939 raft_consensus.cc:493] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.519605 28304 raft_consensus.cc:2468] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.519769 28294 raft_consensus.cc:2468] T c3a49e310b524f3b87eb89f3e84b7800 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.518610 30394 leader_election.cc:290] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.518359 30655 consensus_queue.cc:1048] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.523123 29939 raft_consensus.cc:3060] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.525837 29939 raft_consensus.cc:515] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.526439 29939 leader_election.cc:290] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:46.528208 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1c2d7e4d26994a17bf2d2615d6552412 because the tablet was not found. No further retry: Not found: Tablet not found: 1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:46.528458 28588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.528674 28588 raft_consensus.cc:2468] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
W20260320 15:27:46.528695 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ffe278739bbc4ddab848af9c25241380 because the tablet was not found. No further retry: Not found: Tablet not found: ffe278739bbc4ddab848af9c25241380
I20260320 15:27:46.529265 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6292904782fd4b619483135e58b66684 (table default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]) successfully deleted
I20260320 15:27:46.529537 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b838187c21448b6909ad183f9824d2b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:46.529711 28574 raft_consensus.cc:3060] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:46.529589 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because the tablet was not found. No further retry: Not found: Tablet not found: 2ee4a0d559ce4dfea878bd3f1c03c09a
W20260320 15:27:46.529937 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 533105ad6e054fe3a4db32855bc688bd because the tablet was not found. No further retry: Not found: Tablet not found: 533105ad6e054fe3a4db32855bc688bd
I20260320 15:27:46.530105 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
W20260320 15:27:46.530479 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3ed5d220ec4b406aa5bae970f785de63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ed5d220ec4b406aa5bae970f785de63 already in progress: creating tablet
W20260320 15:27:46.530696 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e779a274913f4caeaf82e3f180457830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e779a274913f4caeaf82e3f180457830 already in progress: creating tablet
I20260320 15:27:46.530892 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:46.531111 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 87cb35cefee841d0881b08c456df5a97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87cb35cefee841d0881b08c456df5a97 already in progress: creating tablet
I20260320 15:27:46.533027 28574 raft_consensus.cc:2468] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:46.533035 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.533267 28446 raft_consensus.cc:2468] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.533476 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.533625 28440 raft_consensus.cc:3060] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.533934 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de834725bda4f249882914f1247ab62" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.534327 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.534484 28439 raft_consensus.cc:3060] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.536445 28736 tablet_bootstrap.cc:492] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.536549 28440 raft_consensus.cc:2468] T 8f408aa18d184e5c9a1f900827e70ca8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.536592 28736 ts_tablet_manager.cc:1403] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.121s user 0.003s sys 0.000s
I20260320 15:27:46.537143 28439 raft_consensus.cc:2468] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.537343 28736 raft_consensus.cc:359] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.537534 28736 raft_consensus.cc:385] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.537578 28736 raft_consensus.cc:740] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.537773 28736 consensus_queue.cc:260] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.538488 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b40dfe348b48ea8d941e3d97f27d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.538661 28736 ts_tablet_manager.cc:1434] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:46.538964 28736 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.541774 30673 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:46.542018 30673 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
W20260320 15:27:46.544440 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet aba79ddde6e842dfb67597c5d9e371cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba79ddde6e842dfb67597c5d9e371cb already in progress: creating tablet
W20260320 15:27:46.544770 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 2e9791ac61b34fc09a8292c79f0c4032 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2e9791ac61b34fc09a8292c79f0c4032 already in progress: creating tablet
I20260320 15:27:46.545116 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008811c90aeb4ca4a4ad5c0190a60dcc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.545538 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5095b42b69d40539118c1f63a95f795" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.545653 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 726 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:46.546720 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]) successfully deleted
I20260320 15:27:46.546765 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "321fe367ec3348f8a049167f3f8c15d5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.547091 30623 raft_consensus.cc:493] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.547245 30623 raft_consensus.cc:515] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.547387 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.547575 28576 raft_consensus.cc:2468] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:46.547842 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:46.547963 28584 raft_consensus.cc:3060] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.551936 30456 raft_consensus.cc:493] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.552104 30456 raft_consensus.cc:515] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.552276 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f4a2c8ebc374de9bd8078a757c6be57" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.552860 30456 leader_election.cc:290] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.553056 28584 raft_consensus.cc:2468] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:46.553555 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.554107 28382 leader_election.cc:343] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.555665 30673 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:46.556831 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.557101 28443 raft_consensus.cc:2393] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:46.557898 30623 leader_election.cc:290] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:46.558583 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.562125 28133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 726 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:46.565948 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 312) took 2757 ms. Trace:
I20260320 15:27:46.569784 29643 rpcz_store.cc:276] 0320 15:27:43.808382 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.808455 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:46.297403 (+2488948us) ts_tablet_manager.cc:1151] Deleting tablet 06cb44b9cc6445cf8324cf52190d6773
0320 15:27:46.297419 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.317690 (+ 20271us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.477435 (+159745us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.565912 (+ 88477us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2488661}
I20260320 15:27:46.570348 29656 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892; no voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:46.571187 29643 tablet_replica.cc:333] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.573144 29643 raft_consensus.cc:2243] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.573788 29643 raft_consensus.cc:2272] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.574870 29643 ts_tablet_manager.cc:1916] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:46.577260 28524 consensus_peers.cc:597] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71ec63fb080545de838803033f204fab. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.577332 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]) successfully deleted
I20260320 15:27:46.584169 30673 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260320 15:27:46.596743 28736 tablet_bootstrap.cc:654] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:46.583176 28521 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260320 15:27:46.597584 28524 consensus_peers.cc:597] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a4b5a658cd84b04b50b54973c065889. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.606959 30655 consensus_queue.cc:1048] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.609829 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 7840b529206d482283cc550d2c5e7606
I20260320 15:27:46.609970 28903 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 49654c67435a48448794879464f6cfb2
I20260320 15:27:46.610423 28892 catalog_manager.cc:6741] ReplaceTablet: tablet 7840b529206d482283cc550d2c5e7606 deleted and replaced by tablet 31ec989bdd1e4e4eadbe011ada114fbf
I20260320 15:27:46.610548 28903 catalog_manager.cc:6741] ReplaceTablet: tablet 49654c67435a48448794879464f6cfb2 deleted and replaced by tablet e44031268eb54d2796040168e7a37e66
I20260320 15:27:46.610708 28903 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 156) took 1006 ms. Trace:
I20260320 15:27:46.610873 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_78c6dd62d8834a39afa05aac3371a7c0" } modify_external_catalogs: true
I20260320 15:27:46.611093 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_78c6dd62d8834a39afa05aac3371a7c0" } modify_external_catalogs: true
I20260320 15:27:46.611292 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.611515 28294 raft_consensus.cc:2468] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:46.610837 28903 rpcz_store.cc:276] 0320 15:27:45.604249 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.484937 (+880688us) service_pool.cc:225] Handling call
0320 15:27:46.493570 (+ 8633us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61ae0394b 00007fe6118f2107 00007fe6118f1e35 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:46.610658 (+117088us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.588248 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.588351 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.588357 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:46.588359 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:46.588456 (+ 97us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.588466 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.588471 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.588553 (+ 82us) tablet.cc:700] Row locks acquired
0320 15:27:46.588556 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.588588 (+ 32us) write_op.cc:270] Start()
0320 15:27:46.588643 (+ 55us) write_op.cc:276] Timestamp: P: 1774020466588544 usec, L: 0
0320 15:27:46.588646 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.588812 (+ 166us) log.cc:844] Serialized 622 byte log entry
0320 15:27:46.596642 (+ 7830us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.597340 (+ 698us) write_op.cc:301] APPLY: starting
0320 15:27:46.597355 (+ 15us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.597393 (+ 38us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.597437 (+ 44us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.597559 (+ 122us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.597629 (+ 70us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:46.597636 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:46.597707 (+ 71us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.597775 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.597801 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:46.597808 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.598255 (+ 447us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.598262 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":18820352,"child_traces":[["op",{"apply.queue_time_us":656,"apply.run_cpu_time_us":557,"apply.run_wall_time_us":824,"mutex_wait_us":92316,"num_ops":2,"prepare.queue_time_us":93427,"prepare.run_cpu_time_us":649,"prepare.run_wall_time_us":987,"raft.queue_time_us":331,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":7973}]]}
I20260320 15:27:46.612177 28903 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_462a9e5f58364d459620a9eec9c2b570" } modify_external_catalogs: true
I20260320 15:27:46.612439 28903 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_462a9e5f58364d459620a9eec9c2b570" } modify_external_catalogs: true
I20260320 15:27:46.615342 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet 49654c67435a48448794879464f6cfb2 with delete_type TABLET_DATA_DELETED (replaced by tablet e44031268eb54d2796040168e7a37e66) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.615451 28889 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 155) took 1088 ms. Trace:
I20260320 15:27:46.616046 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet 7840b529206d482283cc550d2c5e7606 with delete_type TABLET_DATA_DELETED (replaced by tablet 31ec989bdd1e4e4eadbe011ada114fbf) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:46.615583 28889 rpcz_store.cc:276] 0320 15:27:45.526554 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.451085 (+924531us) service_pool.cc:225] Handling call
0320 15:27:46.483413 (+ 32328us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.483969 (+ 556us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.545967 (+ 61998us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.609626 (+ 63659us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.615361 (+ 5735us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.500940 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.501181 (+ 241us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.501192 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:46.501198 (+ 6us) tablet.cc:661] Decoding operations
0320 15:27:46.501431 (+ 233us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.501448 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.501453 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.501556 (+ 103us) tablet.cc:700] Row locks acquired
0320 15:27:46.501558 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:46.501603 (+ 45us) write_op.cc:270] Start()
0320 15:27:46.501640 (+ 37us) write_op.cc:276] Timestamp: P: 1774020466501564 usec, L: 0
0320 15:27:46.501644 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.501864 (+ 220us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.515861 (+ 13997us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.531347 (+ 15486us) write_op.cc:301] APPLY: starting
0320 15:27:46.531375 (+ 28us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.531434 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.531499 (+ 65us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.531698 (+ 199us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.531746 (+ 48us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.531758 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:46.531877 (+ 119us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.531959 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.531986 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:46.531993 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.532078 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.532082 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15360,"apply.run_cpu_time_us":737,"apply.run_wall_time_us":737,"num_ops":2,"prepare.queue_time_us":16397,"prepare.run_cpu_time_us":1058,"prepare.run_wall_time_us":1059,"raft.queue_time_us":503,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":270,"replication_time_us":14162}]]}
I20260320 15:27:46.616801 28889 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_786940b8acbe4d13944ef32c8b08cbba" } new_table_name: "default.Table_0f97e4f818004a0785b29e361c14a453" modify_external_catalogs: true
I20260320 15:27:46.617385 30456 raft_consensus.cc:493] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.617424 28901 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 154) took 1116 ms. Trace:
I20260320 15:27:46.617548 30456 raft_consensus.cc:515] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.617748 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.617551 28901 rpcz_store.cc:276] 0320 15:27:45.500899 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.450403 (+949504us) service_pool.cc:225] Handling call
0320 15:27:46.485376 (+ 34973us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:46.485873 (+ 497us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:46.607885 (+122012us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.608006 (+ 121us) write_op.cc:454] Released schema lock
0320 15:27:46.608848 (+ 842us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:46.617237 (+ 8389us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:46.617344 (+ 107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.587267 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.587430 (+ 163us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.587437 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:46.587440 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.587587 (+ 147us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.587604 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.587610 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.587699 (+ 89us) tablet.cc:700] Row locks acquired
0320 15:27:46.587702 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.587750 (+ 48us) write_op.cc:270] Start()
0320 15:27:46.587804 (+ 54us) write_op.cc:276] Timestamp: P: 1774020466587703 usec, L: 0
0320 15:27:46.587808 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.588026 (+ 218us) log.cc:844] Serialized 433 byte log entry
0320 15:27:46.596542 (+ 8516us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.596673 (+ 131us) write_op.cc:301] APPLY: starting
0320 15:27:46.596696 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.596742 (+ 46us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.596772 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.596950 (+ 178us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.596991 (+ 41us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:46.597000 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:46.597096 (+ 96us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.597196 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.597222 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:46.597230 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":45,"apply.run_cpu_time_us":635,"apply.run_wall_time_us":635,"mutex_wait_us":100532,"num_ops":2,"prepare.queue_time_us":100777,"prepare.run_cpu_time_us":902,"prepare.run_wall_time_us":964,"raft.queue_time_us":728,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":252,"replication_time_us":8680}]]}
I20260320 15:27:46.619386 30456 leader_election.cc:290] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.619808 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 7840b529206d482283cc550d2c5e7606 with delete_type TABLET_DATA_DELETED (replaced by tablet 31ec989bdd1e4e4eadbe011ada114fbf) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.620110 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.620858 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.621588 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.621876 28734 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.622315 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.623752 28734 ts_tablet_manager.cc:1403] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.424s user 0.008s sys 0.023s
I20260320 15:27:46.625535 29678 ts_tablet_manager.cc:1929] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.625700 29678 log.cc:1199] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1bd3032950254e6f9db8da35dbb1acc5
I20260320 15:27:46.626382 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.626343 28734 raft_consensus.cc:359] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.627208 28734 raft_consensus.cc:385] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.627337 28734 raft_consensus.cc:740] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.627874 28734 consensus_queue.cc:260] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.628726 28734 ts_tablet_manager.cc:1434] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:46.628830 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 49654c67435a48448794879464f6cfb2 with delete_type TABLET_DATA_DELETED (replaced by tablet e44031268eb54d2796040168e7a37e66) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:46.629160 28734 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.629810 29678 ts_tablet_manager.cc:1950] T 1bd3032950254e6f9db8da35dbb1acc5 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:46.618423 28901 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_b35323a629d64449b3f893c3049761b6" } modify_external_catalogs: true
I20260320 15:27:46.636075 28901 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_b35323a629d64449b3f893c3049761b6" } modify_external_catalogs: true
I20260320 15:27:46.640091 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 276) took 2831 ms. Trace:
I20260320 15:27:46.640200 29678 rpcz_store.cc:276] 0320 15:27:43.809117 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.809328 (+ 211us) service_pool.cc:225] Handling call
0320 15:27:46.430300 (+2620972us) ts_tablet_manager.cc:1151] Deleting tablet 1bd3032950254e6f9db8da35dbb1acc5
0320 15:27:46.430314 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.584124 (+153810us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.587413 (+ 3289us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.640068 (+ 52655us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2620809}
I20260320 15:27:46.640604 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 278) took 2716 ms. Trace:
I20260320 15:27:46.640690 29678 rpcz_store.cc:276] 0320 15:27:43.923778 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.923878 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:46.640484 (+2716606us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:46.640494 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.640587 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2716294}
I20260320 15:27:46.640950 29678 tablet_replica.cc:333] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:46.641142 29678 raft_consensus.cc:2243] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.641309 29678 raft_consensus.cc:2272] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.642035 29678 ts_tablet_manager.cc:1916] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.648056 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197f1935099140a98c5eb801cf44046a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.650722 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.640029 28733 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:46.651922 28733 ts_tablet_manager.cc:1403] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.163s user 0.003s sys 0.001s
I20260320 15:27:46.648592 29640 rpcz_store.cc:276] 0320 15:27:35.745083 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.746289 (+ 1206us) service_pool.cc:225] Handling call
0320 15:27:45.911180 (+10164891us) ts_tablet_manager.cc:1151] Deleting tablet 6771bd6a3f0e47d99a0191eae25cb2d2
0320 15:27:45.911192 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:45.983735 (+ 72543us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.117971 (+134236us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.203967 (+ 85996us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10164680}
I20260320 15:27:46.653128 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 6771bd6a3f0e47d99a0191eae25cb2d2 (table default.Table_f7415957b95743768b23acf4577d7db3 [id=f8b01008d6e0436dbd213ada814154fe]) successfully deleted
I20260320 15:27:46.657223 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20eec704800e4d8b924d97556cc2700e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.658704 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.659199 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:46.660962 28239 leader_election.cc:343] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 8d92e17cea044541968356ae993be1e7
I20260320 15:27:46.661093 28239 leader_election.cc:304] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.661427 30466 raft_consensus.cc:2749] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.662333 28239 leader_election.cc:304] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.662917 30466 raft_consensus.cc:2749] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.664024 30466 consensus_queue.cc:1048] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.664999 30466 consensus_queue.cc:1048] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.667632 28736 tablet_bootstrap.cc:492] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.667889 28736 ts_tablet_manager.cc:1403] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.129s user 0.000s sys 0.006s
I20260320 15:27:46.668825 28736 raft_consensus.cc:359] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.668972 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 178) took 10897 ms. Trace:
I20260320 15:27:46.669071 28736 raft_consensus.cc:385] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.669143 28736 raft_consensus.cc:740] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.669111 29640 rpcz_store.cc:276] 0320 15:27:35.771322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.771384 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:46.668790 (+10897406us) ts_tablet_manager.cc:1151] Deleting tablet ae3422cf33324ab89a2b44e27cae0eb2
0320 15:27:46.668804 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.668944 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10897113}
I20260320 15:27:46.669365 28736 consensus_queue.cc:260] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.669991 28736 ts_tablet_manager.cc:1434] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260320 15:27:46.670019 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae3422cf33324ab89a2b44e27cae0eb2 already in progress: creating tablet
I20260320 15:27:46.670387 28736 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.670585 29640 tablet_replica.cc:333] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:46.670854 29640 raft_consensus.cc:2243] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:46.671303 28734 tablet_bootstrap.cc:654] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.671941 28736 tablet_bootstrap.cc:654] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.673210 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:46.673739 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 64737.5us
W20260320 15:27:46.673887 28382 leader_election.cc:343] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.674322 30655 consensus_queue.cc:1048] T 44327000792444b89265f4a106eb11d0 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.674834 28733 raft_consensus.cc:359] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.677963 28733 raft_consensus.cc:385] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.678068 28733 raft_consensus.cc:740] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.678320 28733 consensus_queue.cc:260] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:46.679983 28524 consensus_peers.cc:597] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cea3bf14653f4555acfed1577a79ee4f. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.682200 28733 ts_tablet_manager.cc:1434] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.030s user 0.000s sys 0.002s
I20260320 15:27:46.682551 28733 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:46.683633 28524 leader_election.cc:343] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.685998 28733 tablet_bootstrap.cc:654] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.675151 29640 raft_consensus.cc:2272] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.690053 29640 ts_tablet_manager.cc:1916] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.690004 30655 consensus_queue.cc:1048] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.696105 29643 ts_tablet_manager.cc:1929] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.696776 30597 catalog_manager.cc:1269] Loaded cluster ID: 4506d4959d614cc6be6dbef586c372b2
I20260320 15:27:46.707422 30655 consensus_queue.cc:1048] T 7e13c75a376c47c9a0083173865ce4fa P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.707834 28524 leader_election.cc:304] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.707873 29939 consensus_queue.cc:1048] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.708551 30597 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: loading cluster ID for follower catalog manager: success
I20260320 15:27:46.709910 29939 raft_consensus.cc:2749] T 9b838187c21448b6909ad183f9824d2b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.710795 29643 log.cc:1199] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1bd3032950254e6f9db8da35dbb1acc5
I20260320 15:27:46.711335 29643 ts_tablet_manager.cc:1950] T 1bd3032950254e6f9db8da35dbb1acc5 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.666306 28239 leader_election.cc:304] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:46.710109 28524 leader_election.cc:343] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.711860 30466 raft_consensus.cc:2749] T 44327000792444b89265f4a106eb11d0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:46.712189 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 46036.9us
W20260320 15:27:46.713078 28524 leader_election.cc:343] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.713618 29939 consensus_queue.cc:1048] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.714745 28239 leader_election.cc:304] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.715013 30622 raft_consensus.cc:2749] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.715639 28239 leader_election.cc:304] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:46.715921 28239 raft_consensus.cc:2686] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:46.718472 28734 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.718683 28734 ts_tablet_manager.cc:1403] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.090s user 0.001s sys 0.005s
I20260320 15:27:46.718844 28239 leader_election.cc:304] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.719183 30622 raft_consensus.cc:2749] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.725154 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 313) took 2916 ms. Trace:
I20260320 15:27:46.725291 28734 raft_consensus.cc:359] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.725523 28734 raft_consensus.cc:385] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.725425 29678 ts_tablet_manager.cc:1929] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:46.725579 28734 raft_consensus.cc:740] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.725742 29678 log.cc:1199] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/41da3cb06b1147fbae2695bc5c9ffaa5
I20260320 15:27:46.725301 29643 rpcz_store.cc:276] 0320 15:27:43.808546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.808624 (+ 78us) service_pool.cc:225] Handling call
0320 15:27:46.571121 (+2762497us) ts_tablet_manager.cc:1151] Deleting tablet 1bd3032950254e6f9db8da35dbb1acc5
0320 15:27:46.571134 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.605399 (+ 34265us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.695964 (+ 90565us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.725121 (+ 29157us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43,"tablet-delete.queue_time_us":2762375}
I20260320 15:27:46.726342 29678 ts_tablet_manager.cc:1950] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:46.726347 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1bd3032950254e6f9db8da35dbb1acc5 (table default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]) successfully deleted
I20260320 15:27:46.726423 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 314) took 2855 ms. Trace:
I20260320 15:27:46.726497 29643 rpcz_store.cc:276] 0320 15:27:43.871436 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.871496 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:46.726298 (+2854802us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:46.726311 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.726403 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2854464}
W20260320 15:27:46.726950 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e7bd0616f5604f8fbb8edd8e9c230f39 because the tablet was not found. No further retry: Not found: Tablet not found: e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:46.727308 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 315) took 2855 ms. Trace:
I20260320 15:27:46.727399 29643 rpcz_store.cc:276] 0320 15:27:43.884495 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.884593 (+ 98us) service_pool.cc:225] Handling call
0320 15:27:46.727213 (+2842620us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:46.727223 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.727291 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2842350}
W20260320 15:27:46.727941 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because the tablet was not found. No further retry: Not found: Tablet not found: 7394e941c99e4358ad7a33db84b15348
I20260320 15:27:46.728299 28734 consensus_queue.cc:260] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.728715 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 317) took 2843 ms. Trace:
I20260320 15:27:46.728804 29643 rpcz_store.cc:276] 0320 15:27:43.884827 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.884915 (+ 88us) service_pool.cc:225] Handling call
0320 15:27:46.728618 (+2843703us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:46.728630 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.728700 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2843544}
W20260320 15:27:46.729202 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c3bc4dd47817490d88725127b456cc70 because the tablet was not found. No further retry: Not found: Tablet not found: c3bc4dd47817490d88725127b456cc70
I20260320 15:27:46.729921 28734 ts_tablet_manager.cc:1434] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:46.731382 29939 consensus_queue.cc:1048] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:46.732199 28524 leader_election.cc:343] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.737488 29643 tablet_replica.cc:333] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:46.737706 29643 raft_consensus.cc:2243] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.737877 29643 raft_consensus.cc:2272] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.738420 29643 ts_tablet_manager.cc:1916] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.739164 28524 leader_election.cc:304] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.739921 29939 raft_consensus.cc:2804] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.740010 29939 raft_consensus.cc:493] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.740068 29939 raft_consensus.cc:3060] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.745198 28524 leader_election.cc:304] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.745611 30655 raft_consensus.cc:2804] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.747653 30655 raft_consensus.cc:697] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.747943 30655 consensus_queue.cc:237] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.748292 28524 leader_election.cc:304] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.748601 30629 raft_consensus.cc:2804] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.748757 30629 raft_consensus.cc:697] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.737244 28239 leader_election.cc:304] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.751963 28239 leader_election.cc:304] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.752151 30623 raft_consensus.cc:2749] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.752352 30622 raft_consensus.cc:2749] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.752597 28239 leader_election.cc:304] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.752836 30622 raft_consensus.cc:2749] T 321fe367ec3348f8a049167f3f8c15d5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.753015 28239 leader_election.cc:304] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:46.753212 30622 raft_consensus.cc:2749] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:46.744643 28736 tablet_bootstrap.cc:492] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.753895 28736 ts_tablet_manager.cc:1403] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.000s
I20260320 15:27:46.754175 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet d823ab41194542fba5540e53e9f4680b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.755007 28736 raft_consensus.cc:359] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.755226 28736 raft_consensus.cc:385] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.755306 28736 raft_consensus.cc:740] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.755508 28736 consensus_queue.cc:260] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.756184 28736 ts_tablet_manager.cc:1434] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:46.756675 28736 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:46.758054 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 279) took 2812 ms. Trace:
I20260320 15:27:46.758147 29678 rpcz_store.cc:276] 0320 15:27:43.945478 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.945567 (+ 89us) service_pool.cc:225] Handling call
0320 15:27:46.640904 (+2695337us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:46.640912 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.701019 (+ 60107us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.725178 (+ 24159us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.758033 (+ 32855us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2695082}
I20260320 15:27:46.758482 29678 tablet_replica.cc:333] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:46.758669 29678 raft_consensus.cc:2243] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:46.758941 29678 raft_consensus.cc:2272] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.762836 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.765583 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet b4cad900411e4c9686701aa99206dd23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.766161 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 9d7f7b99899c4001adcbdce58910e717 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.766646 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d7f7b99899c4001adcbdce58910e717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.766875 30597 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: acquiring CA information for follower catalog manager: success
I20260320 15:27:46.767360 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 806d635bb5594f0bb826b9b31468b485 (DEFAULT_TABLE table=default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.767455 28271 tablet_service.cc:1511] Processing CreateTablet for tablet 9a2003e2af9a4d9c854990e1db41af28 (DEFAULT_TABLE table=default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.767766 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d635bb5594f0bb826b9b31468b485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.767962 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a2003e2af9a4d9c854990e1db41af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.770335 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet db16d86664794305bde714a7ff4d5caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.771003 28278 tablet_service.cc:1511] Processing CreateTablet for tablet f77a32615c954da581e9a9e4cb45c497 (DEFAULT_TABLE table=default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.771410 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f77a32615c954da581e9a9e4cb45c497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.771806 30394 consensus_queue.cc:1048] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.772763 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 7840b529206d482283cc550d2c5e7606 with delete_type TABLET_DATA_DELETED (replaced by tablet 31ec989bdd1e4e4eadbe011ada114fbf) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.773162 28266 tablet_service.cc:1558] Processing DeleteTablet for tablet 49654c67435a48448794879464f6cfb2 with delete_type TABLET_DATA_DELETED (replaced by tablet e44031268eb54d2796040168e7a37e66) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.773288 28276 tablet_service.cc:1511] Processing CreateTablet for tablet a8b40dfe348b48ea8d941e3d97f27d73 (DEFAULT_TABLE table=default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:46.773687 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8b40dfe348b48ea8d941e3d97f27d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:46.774389 28383 leader_election.cc:304] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.775122 30636 raft_consensus.cc:2764] T ea7f260f4edd4fc082ab82880143fa31 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260320 15:27:46.775713 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1bd3032950254e6f9db8da35dbb1acc5 (table default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]) successfully deleted
W20260320 15:27:46.776008 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because the tablet was not found. No further retry: Not found: Tablet not found: 0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:46.776587 28736 tablet_bootstrap.cc:654] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.777163 28584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c7a605bdcf14d2ab64ad078efd3d999" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:46.778977 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:46.802192 28734 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:46.803426 29939 raft_consensus.cc:515] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.804234 29939 leader_election.cc:290] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.807859 29640 ts_tablet_manager.cc:1929] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.812355 29678 ts_tablet_manager.cc:1916] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.808022 29640 log.cc:1199] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/71ec63fb080545de838803033f204fab
I20260320 15:27:46.812893 29640 ts_tablet_manager.cc:1950] T 71ec63fb080545de838803033f204fab P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:46.813714 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 179) took 11041 ms. Trace:
I20260320 15:27:46.813812 29640 rpcz_store.cc:276] 0320 15:27:35.772399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.772452 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:46.670383 (+10897931us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:46.670395 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.693349 (+ 22954us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.807717 (+114368us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.813689 (+ 5972us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6550,"tablet-delete.queue_time_us":10897713}
I20260320 15:27:46.814388 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 71ec63fb080545de838803033f204fab (table default.Table_82d375c404eb4fdca024e9fc85f87e1a [id=c987de7f081a4811bb4698cd8a890e88]) successfully deleted
I20260320 15:27:46.815452 28734 tablet_bootstrap.cc:654] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:46.831220 30629 consensus_queue.cc:237] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.835438 30597 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260320 15:27:46.852339 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 77513.3us
I20260320 15:27:46.858553 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:46.869500 28443 raft_consensus.cc:3060] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.872430 28443 raft_consensus.cc:2468] T 31379c9e2adc4fd48e0d0b6e9443615a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:46.873323 28383 leader_election.cc:343] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.873946 28383 leader_election.cc:304] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.876552 30636 raft_consensus.cc:2749] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:46.877130 28383 leader_election.cc:343] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.877350 28383 leader_election.cc:304] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.878086 28736 tablet_bootstrap.cc:492] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:46.878271 28736 ts_tablet_manager.cc:1403] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.122s user 0.001s sys 0.002s
I20260320 15:27:46.879249 28736 raft_consensus.cc:359] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.881460 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:46.883010 30636 raft_consensus.cc:2749] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:46.883669 28383 leader_election.cc:343] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.885563 28383 leader_election.cc:304] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.886255 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49654c67435a48448794879464f6cfb2" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.887027 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:46.887248 28303 raft_consensus.cc:2468] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:46.888348 30636 raft_consensus.cc:2749] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:46.889045 28383 leader_election.cc:343] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.892210 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f408aa18d184e5c9a1f900827e70ca8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.892403 28733 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:46.892419 28303 raft_consensus.cc:3060] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.892552 28733 ts_tablet_manager.cc:1403] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.210s user 0.008s sys 0.001s
W20260320 15:27:46.894833 28524 consensus_peers.cc:597] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41da3cb06b1147fbae2695bc5c9ffaa5. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.895011 28524 leader_election.cc:304] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.895273 28303 raft_consensus.cc:2468] T 8f408aa18d184e5c9a1f900827e70ca8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.895301 30630 raft_consensus.cc:2804] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:46.895408 30630 raft_consensus.cc:697] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:46.895681 30630 consensus_queue.cc:237] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.896202 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea7f260f4edd4fc082ab82880143fa31" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.896456 28303 raft_consensus.cc:3060] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.899760 28303 raft_consensus.cc:2468] T ea7f260f4edd4fc082ab82880143fa31 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.900750 30623 consensus_queue.cc:1048] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:46.879492 28736 raft_consensus.cc:385] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.901058 28736 raft_consensus.cc:740] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.901399 28736 consensus_queue.cc:260] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.902011 28736 ts_tablet_manager.cc:1434] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260320 15:27:46.902313 28736 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:46.915486 28383 leader_election.cc:343] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.886186 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
I20260320 15:27:46.917423 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:46.918843 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:46.920498 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:46.922780 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:46.923105 29678 ts_tablet_manager.cc:1929] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:46.923233 29678 log.cc:1199] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:46.924847 29643 ts_tablet_manager.cc:1929] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260320 15:27:46.925002 28383 leader_election.cc:343] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:46.925412 28383 leader_election.cc:304] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:46.931382 29640 tablet_replica.cc:333] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:46.931823 29640 raft_consensus.cc:2243] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:46.932515 29640 raft_consensus.cc:2272] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:46.934648 29640 ts_tablet_manager.cc:1916] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:46.925022 29643 log.cc:1199] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6292904782fd4b619483135e58b66684
I20260320 15:27:46.948544 29643 ts_tablet_manager.cc:1950] T 6292904782fd4b619483135e58b66684 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:46.935065 28733 raft_consensus.cc:359] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.950840 28733 raft_consensus.cc:385] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:46.950995 28733 raft_consensus.cc:740] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:46.951335 28733 consensus_queue.cc:260] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.954273 30355 raft_consensus.cc:493] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:46.954437 30355 raft_consensus.cc:515] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.955143 30355 leader_election.cc:290] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.957320 28733 ts_tablet_manager.cc:1434] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.065s user 0.002s sys 0.001s
I20260320 15:27:46.957736 28733 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:46.960580 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:46.960865 28447 raft_consensus.cc:2468] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:46.962046 28524 consensus_peers.cc:597] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bd3032950254e6f9db8da35dbb1acc5. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:46.962400 28524 leader_election.cc:304] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:46.962953 30394 raft_consensus.cc:2804] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:46.963047 30394 raft_consensus.cc:493] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:46.963125 30394 raft_consensus.cc:3060] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.967296 30394 raft_consensus.cc:515] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:46.967655 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31379c9e2adc4fd48e0d0b6e9443615a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:46.967847 28298 raft_consensus.cc:3060] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:46.968286 30394 leader_election.cc:290] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:46.970942 28298 raft_consensus.cc:2468] T 31379c9e2adc4fd48e0d0b6e9443615a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:46.977021 28734 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:46.977216 28734 ts_tablet_manager.cc:1403] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.175s user 0.002s sys 0.002s
I20260320 15:27:47.000314 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:47.012418 28239 leader_election.cc:304] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.012785 30623 raft_consensus.cc:2749] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:47.013244 30622 consensus_queue.cc:1048] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.013883 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.029433 29678 ts_tablet_manager.cc:1950] T dd315b5d49104d679b03515b65ee30ac P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.030407 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 280) took 3084 ms. Trace:
I20260320 15:27:47.030525 29678 rpcz_store.cc:276] 0320 15:27:43.945757 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.945993 (+ 236us) service_pool.cc:225] Handling call
0320 15:27:46.758435 (+2812442us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:46.758445 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.919054 (+160609us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.922977 (+ 3923us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.030376 (+107399us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2812258}
I20260320 15:27:47.030869 29678 tablet_replica.cc:333] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.031275 28736 tablet_bootstrap.cc:654] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.040607 29678 raft_consensus.cc:2243] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.040854 29678 raft_consensus.cc:2272] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.042788 29678 ts_tablet_manager.cc:1916] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.073025 30622 raft_consensus.cc:493] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.073295 30622 raft_consensus.cc:515] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.074303 30622 leader_election.cc:290] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:47.074928 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.075440 29678 ts_tablet_manager.cc:1929] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:47.075835 29678 log.cc:1199] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/889d620c311a42039c22af53ad0802f2
I20260320 15:27:47.076275 29678 ts_tablet_manager.cc:1950] T 889d620c311a42039c22af53ad0802f2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.076839 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57cbefe3a19044f78b64db951df5f7c1" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.077111 28447 raft_consensus.cc:2393] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:47.077139 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 282) took 3131 ms. Trace:
I20260320 15:27:47.077229 29678 rpcz_store.cc:276] 0320 15:27:43.946052 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.946329 (+ 277us) service_pool.cc:225] Handling call
0320 15:27:47.030815 (+3084486us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:47.030827 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.057419 (+ 26592us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.069324 (+ 11905us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.077117 (+ 7793us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1278,"tablet-delete.queue_time_us":3084317}
I20260320 15:27:47.077873 28240 leader_election.cc:304] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.078410 30623 raft_consensus.cc:3060] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.081848 30623 raft_consensus.cc:2749] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.085970 30395 raft_consensus.cc:493] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.086246 30395 raft_consensus.cc:515] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.089919 29678 tablet_replica.cc:333] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.090174 29678 raft_consensus.cc:2243] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.090523 29678 pending_rounds.cc:70] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:47.090574 29678 pending_rounds.cc:77] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387817937149952 op_type: NO_OP noop_request { }
I20260320 15:27:47.090668 29678 raft_consensus.cc:2889] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:47.090751 29678 raft_consensus.cc:2272] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.090655 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:47.090871 28447 raft_consensus.cc:3060] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.091493 30395 leader_election.cc:290] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.094810 28447 raft_consensus.cc:2468] T 11bef661385b4221a991710ffb34dccb P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:47.096664 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bef661385b4221a991710ffb34dccb" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:47.098604 29678 ts_tablet_manager.cc:1916] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.041217 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet dd315b5d49104d679b03515b65ee30ac (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
W20260320 15:27:47.124357 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 83104.1us
I20260320 15:27:47.125062 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 889d620c311a42039c22af53ad0802f2 (table default.Table_bc31ab0980a04953909f5c8d56ae39ee [id=60bade2572aa452d9c6a06374da38306]) successfully deleted
I20260320 15:27:47.126049 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:47.129667 29678 ts_tablet_manager.cc:1929] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.129834 29678 log.cc:1199] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/56fa691a26114657ad768545a30dfd3c
I20260320 15:27:47.130218 29678 ts_tablet_manager.cc:1950] T 56fa691a26114657ad768545a30dfd3c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.130162 30636 raft_consensus.cc:2749] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.131063 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 281) took 3185 ms. Trace:
I20260320 15:27:47.131145 28734 raft_consensus.cc:359] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.131184 29678 rpcz_store.cc:276] 0320 15:27:43.945896 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.946936 (+ 1040us) service_pool.cc:225] Handling call
0320 15:27:47.089844 (+3142908us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:47.089862 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.102486 (+ 12624us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.129522 (+ 27036us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.131037 (+ 1515us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3142735}
I20260320 15:27:47.131367 28734 raft_consensus.cc:385] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.131429 28734 raft_consensus.cc:740] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.131705 28734 consensus_queue.cc:260] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.131826 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 283) took 3184 ms. Trace:
I20260320 15:27:47.131918 29678 rpcz_store.cc:276] 0320 15:27:43.947222 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.947290 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:47.131701 (+3184411us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:47.131711 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.131809 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3184249}
I20260320 15:27:47.132263 28734 ts_tablet_manager.cc:1434] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.155s user 0.000s sys 0.002s
I20260320 15:27:47.132552 28734 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.134217 28734 tablet_bootstrap.cc:654] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.135914 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 284) took 3188 ms. Trace:
I20260320 15:27:47.136029 29678 rpcz_store.cc:276] 0320 15:27:43.947618 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.947689 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:47.135766 (+3188077us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:47.135780 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.135893 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3187907}
I20260320 15:27:47.136569 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 285) took 3163 ms. Trace:
I20260320 15:27:47.136646 29678 rpcz_store.cc:276] 0320 15:27:43.973613 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.973700 (+ 87us) service_pool.cc:225] Handling call
0320 15:27:47.136463 (+3162763us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:47.136473 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.136552 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3162544}
I20260320 15:27:47.138929 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 286) took 3165 ms. Trace:
I20260320 15:27:47.139057 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 316) took 3254 ms. Trace:
I20260320 15:27:47.139104 29678 rpcz_store.cc:276] 0320 15:27:43.973794 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.973960 (+ 166us) service_pool.cc:225] Handling call
0320 15:27:47.138842 (+3164882us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:47.138852 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.138914 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3164715}
I20260320 15:27:47.139150 29643 rpcz_store.cc:276] 0320 15:27:43.884713 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.885037 (+ 324us) service_pool.cc:225] Handling call
0320 15:27:46.730152 (+2845115us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:46.737405 (+ 7253us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:46.741569 (+ 4164us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:46.924747 (+183178us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.139032 (+214285us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":189113,"tablet-delete.queue_time_us":2844993}
I20260320 15:27:47.139642 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 287) took 3165 ms. Trace:
I20260320 15:27:47.139748 29678 rpcz_store.cc:276] 0320 15:27:43.973975 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.974158 (+ 183us) service_pool.cc:225] Handling call
0320 15:27:47.139442 (+3165284us) ts_tablet_manager.cc:1151] Deleting tablet aafd1e123cd043e7828140d2443e047f
0320 15:27:47.139451 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.139625 (+ 174us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3165097}
I20260320 15:27:47.140144 29678 tablet_replica.cc:333] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.140345 29678 raft_consensus.cc:2243] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.140523 29678 raft_consensus.cc:2272] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.141121 29678 ts_tablet_manager.cc:1916] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.141726 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 318) took 3250 ms. Trace:
I20260320 15:27:47.141832 29643 rpcz_store.cc:276] 0320 15:27:43.890925 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.890984 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:47.141587 (+3250603us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:47.141601 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.141700 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3250330}
W20260320 15:27:47.146773 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 221754us
I20260320 15:27:47.131744 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 56fa691a26114657ad768545a30dfd3c (table default.Table_6673a2f894aa425bb079ad34bf6940b9 [id=cff67c42124c45438d21df1676c82c11]) successfully deleted
I20260320 15:27:47.153293 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6292904782fd4b619483135e58b66684 (table default.Table_d7023c991b824e1a87f6fc910db487bd [id=f366689aa2e9476ebae11ee60a988ac4]) successfully deleted
W20260320 15:27:47.153642 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 757b5afdea674217909610e3719a8804 because the tablet was not found. No further retry: Not found: Tablet not found: 757b5afdea674217909610e3719a8804
W20260320 15:27:47.155633 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e7bd0616f5604f8fbb8edd8e9c230f39 because the tablet was not found. No further retry: Not found: Tablet not found: e7bd0616f5604f8fbb8edd8e9c230f39
W20260320 15:27:47.155875 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because the tablet was not found. No further retry: Not found: Tablet not found: b6201b74fdb0428e98f2eebb0b24d01a
W20260320 15:27:47.156025 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because the tablet was not found. No further retry: Not found: Tablet not found: ecaa08ba34514c5e99a9e2d4ba9268a9
W20260320 15:27:47.156183 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c3bc4dd47817490d88725127b456cc70 because the tablet was not found. No further retry: Not found: Tablet not found: c3bc4dd47817490d88725127b456cc70
W20260320 15:27:47.156325 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet aafd1e123cd043e7828140d2443e047f because the tablet was not found. No further retry: Not found: Tablet not found: aafd1e123cd043e7828140d2443e047f
I20260320 15:27:47.172495 29643 tablet_replica.cc:333] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:47.172855 28524 leader_election.cc:304] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.181404 29643 raft_consensus.cc:2243] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.187652 30651 raft_consensus.cc:2749] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.188199 29678 ts_tablet_manager.cc:1929] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260320 15:27:47.188161 28524 consensus_peers.cc:597] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41da3cb06b1147fbae2695bc5c9ffaa5. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:47.188342 29678 log.cc:1199] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/3ccbf4f32636463f8e1e005a629b92cc
I20260320 15:27:47.188551 30651 consensus_queue.cc:1048] T 85c397eab0ed4df188b0dc2a58077b39 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.188805 29678 ts_tablet_manager.cc:1950] T 3ccbf4f32636463f8e1e005a629b92cc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.189793 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 288) took 3215 ms. Trace:
I20260320 15:27:47.189891 29678 rpcz_store.cc:276] 0320 15:27:43.974292 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.974455 (+ 163us) service_pool.cc:225] Handling call
0320 15:27:47.140086 (+3165631us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:47.140094 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.163161 (+ 23067us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.188071 (+ 24910us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.189769 (+ 1698us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3165490}
I20260320 15:27:47.190460 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 289) took 3215 ms. Trace:
I20260320 15:27:47.190558 29678 rpcz_store.cc:276] 0320 15:27:43.974547 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.974671 (+ 124us) service_pool.cc:225] Handling call
0320 15:27:47.190345 (+3215674us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:47.190356 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.190444 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3215505}
I20260320 15:27:47.191314 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 290) took 3216 ms. Trace:
I20260320 15:27:47.191404 29678 rpcz_store.cc:276] 0320 15:27:43.974893 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.974965 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:47.190910 (+3215945us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:47.190919 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.191295 (+ 376us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3215663}
I20260320 15:27:47.191893 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 292) took 3216 ms. Trace:
I20260320 15:27:47.191978 29678 rpcz_store.cc:276] 0320 15:27:43.975215 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.975425 (+ 210us) service_pool.cc:225] Handling call
0320 15:27:47.191762 (+3216337us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:47.191772 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.191877 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3216129}
I20260320 15:27:47.192413 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 295) took 3216 ms. Trace:
I20260320 15:27:47.192497 29678 rpcz_store.cc:276] 0320 15:27:43.975495 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.975589 (+ 94us) service_pool.cc:225] Handling call
0320 15:27:47.192295 (+3216706us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:47.192304 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.192396 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3216531}
I20260320 15:27:47.192956 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 293) took 3217 ms. Trace:
I20260320 15:27:47.193069 29678 rpcz_store.cc:276] 0320 15:27:43.975320 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.975668 (+ 348us) service_pool.cc:225] Handling call
0320 15:27:47.192824 (+3217156us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:47.192833 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.192907 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3217034}
I20260320 15:27:47.193605 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 291) took 3218 ms. Trace:
I20260320 15:27:47.193728 29678 rpcz_store.cc:276] 0320 15:27:43.975016 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.975818 (+ 802us) service_pool.cc:225] Handling call
0320 15:27:47.193398 (+3217580us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:47.193407 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.193485 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3217433}
I20260320 15:27:47.194177 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 296) took 3218 ms. Trace:
I20260320 15:27:47.194291 29678 rpcz_store.cc:276] 0320 15:27:43.975869 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.975973 (+ 104us) service_pool.cc:225] Handling call
0320 15:27:47.194046 (+3218073us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:47.194055 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.194125 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3217970}
I20260320 15:27:47.187124 28733 tablet_bootstrap.cc:700] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Time spent opening tablet: real 0.228s user 0.002s sys 0.000s
I20260320 15:27:47.198416 28733 tablet_bootstrap.cc:654] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.198454 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 294) took 3223 ms. Trace:
I20260320 15:27:47.198640 29678 rpcz_store.cc:276] 0320 15:27:43.975409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976110 (+ 701us) service_pool.cc:225] Handling call
0320 15:27:47.198238 (+3222128us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:47.198253 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.198407 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3221982}
I20260320 15:27:47.199186 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 297) took 3223 ms. Trace:
I20260320 15:27:47.199323 29678 rpcz_store.cc:276] 0320 15:27:43.976109 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976269 (+ 160us) service_pool.cc:225] Handling call
0320 15:27:47.199066 (+3222797us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:47.199076 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.199152 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3222696}
I20260320 15:27:47.200428 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 3ccbf4f32636463f8e1e005a629b92cc (table default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]) successfully deleted
W20260320 15:27:47.200793 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 757b5afdea674217909610e3719a8804 because the tablet was not found. No further retry: Not found: Tablet not found: 757b5afdea674217909610e3719a8804
W20260320 15:27:47.200994 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because the tablet was not found. No further retry: Not found: Tablet not found: 583ed053b2a742c7b029f348f4495101
W20260320 15:27:47.201175 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 5fa5f6fe801b4452bf294a9ae23bb163 because the tablet was not found. No further retry: Not found: Tablet not found: 5fa5f6fe801b4452bf294a9ae23bb163
W20260320 15:27:47.201354 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ad4a30476e7146afae1e77dad078136e because the tablet was not found. No further retry: Not found: Tablet not found: ad4a30476e7146afae1e77dad078136e
W20260320 15:27:47.201503 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8a18f32c29224bd883ceb20658dff228 because the tablet was not found. No further retry: Not found: Tablet not found: 8a18f32c29224bd883ceb20658dff228
W20260320 15:27:47.201678 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because the tablet was not found. No further retry: Not found: Tablet not found: ea6ff59d2ffe4d579e6db2375eb11414
W20260320 15:27:47.201825 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 442ca186c9d4449dade39271dec5c608 because the tablet was not found. No further retry: Not found: Tablet not found: 442ca186c9d4449dade39271dec5c608
W20260320 15:27:47.201992 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8dc27a4968ca4203b67dd3064fb03868 because the tablet was not found. No further retry: Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
W20260320 15:27:47.202196 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 5438fdc6e6944e79ac8aecd0ef774467 because the tablet was not found. No further retry: Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:47.204766 30707 raft_consensus.cc:493] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ee1caebf640d4ec0bf9b28c4f496ca18)
I20260320 15:27:47.205032 30707 raft_consensus.cc:515] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } }
I20260320 15:27:47.205935 30707 leader_election.cc:290] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ee1caebf640d4ec0bf9b28c4f496ca18 (127.27.74.62:43951), 336af1a126ca4ab583c03527b890239a (127.27.74.60:42563)
I20260320 15:27:47.208400 28134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 747 } } ignore_live_leader: false dest_uuid: "336af1a126ca4ab583c03527b890239a" is_pre_election: true
I20260320 15:27:47.210026 28927 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07e8bb8819eb4cb4892e0cba1f758892" candidate_term: 6 candidate_status { last_received { term: 5 index: 747 } } ignore_live_leader: false dest_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" is_pre_election: true
I20260320 15:27:47.211082 29657 leader_election.cc:304] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e8bb8819eb4cb4892e0cba1f758892; no voters: 336af1a126ca4ab583c03527b890239a, ee1caebf640d4ec0bf9b28c4f496ca18
I20260320 15:27:47.199787 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 298) took 3223 ms. Trace:
I20260320 15:27:47.211406 29678 rpcz_store.cc:276] 0320 15:27:43.976272 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976409 (+ 137us) service_pool.cc:225] Handling call
0320 15:27:47.199694 (+3223285us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:47.199701 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.199758 (+ 57us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3223130}
W20260320 15:27:47.211985 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 796c907919df47bf9369e45a1248f96b because the tablet was not found. No further retry: Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b
I20260320 15:27:47.212131 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 301) took 3235 ms. Trace:
I20260320 15:27:47.212235 29678 rpcz_store.cc:276] 0320 15:27:43.976557 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976601 (+ 44us) service_pool.cc:225] Handling call
0320 15:27:47.211973 (+3235372us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:47.211988 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.212080 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3235237}
I20260320 15:27:47.212333 29643 raft_consensus.cc:2272] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:47.212625 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because the tablet was not found. No further retry: Not found: Tablet not found: 43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:47.212962 29643 ts_tablet_manager.cc:1916] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.213577 30707 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 07e8bb8819eb4cb4892e0cba1f758892 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260320 15:27:47.214577 28734 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.214738 28734 ts_tablet_manager.cc:1403] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.082s user 0.007s sys 0.013s
I20260320 15:27:47.215664 28734 raft_consensus.cc:359] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.215871 28734 raft_consensus.cc:385] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.215929 28734 raft_consensus.cc:740] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.200197 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:47.215996 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 304) took 3239 ms. Trace:
I20260320 15:27:47.216126 29678 rpcz_store.cc:276] 0320 15:27:43.976797 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976834 (+ 37us) service_pool.cc:225] Handling call
0320 15:27:47.215809 (+3238975us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:47.215825 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.215949 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3238833}
I20260320 15:27:47.216202 28734 consensus_queue.cc:260] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:47.216674 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7f0242a7968541f480367b45cda5d9cc because the tablet was not found. No further retry: Not found: Tablet not found: 7f0242a7968541f480367b45cda5d9cc
I20260320 15:27:47.216735 28734 ts_tablet_manager.cc:1434] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.217041 28734 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.218070 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:47.219156 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:47.219668 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:47.219993 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:47.220366 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:47.220793 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:47.222940 28734 tablet_bootstrap.cc:654] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.223886 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 303) took 3247 ms. Trace:
I20260320 15:27:47.224048 29678 rpcz_store.cc:276] 0320 15:27:43.976717 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.976930 (+ 213us) service_pool.cc:225] Handling call
0320 15:27:47.223697 (+3246767us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:47.223713 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.223825 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3246636}
W20260320 15:27:47.228969 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7feb8477ebf841a19977a434f365f537 because the tablet was not found. No further retry: Not found: Tablet not found: 7feb8477ebf841a19977a434f365f537
I20260320 15:27:47.229151 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 300) took 3252 ms. Trace:
I20260320 15:27:47.229260 29678 rpcz_store.cc:276] 0320 15:27:43.976490 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.977073 (+ 583us) service_pool.cc:225] Handling call
0320 15:27:47.228947 (+3251874us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:47.228963 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.229103 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3251757}
W20260320 15:27:47.229836 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because the tablet was not found. No further retry: Not found: Tablet not found: 3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:47.229981 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 302) took 3253 ms. Trace:
I20260320 15:27:47.230094 29678 rpcz_store.cc:276] 0320 15:27:43.976635 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.977265 (+ 630us) service_pool.cc:225] Handling call
0320 15:27:47.229819 (+3252554us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:47.229833 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.229934 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3252416}
W20260320 15:27:47.230581 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9176e734212849aab8d99473de02cc79 because the tablet was not found. No further retry: Not found: Tablet not found: 9176e734212849aab8d99473de02cc79
I20260320 15:27:47.231312 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 306) took 3238 ms. Trace:
W20260320 15:27:47.225016 28383 leader_election.cc:343] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.231743 28383 leader_election.cc:304] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.231434 29678 rpcz_store.cc:276] 0320 15:27:43.992641 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.992727 (+ 86us) service_pool.cc:225] Handling call
0320 15:27:47.231166 (+3238439us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:47.231179 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.231267 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3238166}
W20260320 15:27:47.232180 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7be03035dc3d44f9aa5b43b49281a940 because the tablet was not found. No further retry: Not found: Tablet not found: 7be03035dc3d44f9aa5b43b49281a940
I20260320 15:27:47.232265 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 307) took 3239 ms. Trace:
I20260320 15:27:47.232256 30687 raft_consensus.cc:2749] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.232364 29678 rpcz_store.cc:276] 0320 15:27:43.993046 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.993140 (+ 94us) service_pool.cc:225] Handling call
0320 15:27:47.232139 (+3238999us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:47.232152 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.232230 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3238781}
W20260320 15:27:47.232764 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ef09e4afdd1f4fce9caec4dcafa2c50c because the tablet was not found. No further retry: Not found: Tablet not found: ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:47.232842 29678 tablet_replica.cc:333] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.233044 29678 raft_consensus.cc:2243] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.233413 29678 raft_consensus.cc:2272] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.234194 29678 ts_tablet_manager.cc:1916] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:47.236936 28383 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.237349 28383 leader_election.cc:304] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.237636 30687 raft_consensus.cc:2749] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:47.238242 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 49770.1us
I20260320 15:27:47.248407 29678 ts_tablet_manager.cc:1929] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.248581 29678 log.cc:1199] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/6d9b0f1936f54c699211c5ee9709f9ed
I20260320 15:27:47.248996 29678 ts_tablet_manager.cc:1950] T 6d9b0f1936f54c699211c5ee9709f9ed P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.249951 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 308) took 3256 ms. Trace:
I20260320 15:27:47.250092 29678 rpcz_store.cc:276] 0320 15:27:43.993465 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.993547 (+ 82us) service_pool.cc:225] Handling call
0320 15:27:47.232744 (+3239197us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:47.232756 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.244849 (+ 12093us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.248234 (+ 3385us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.249900 (+ 1666us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3239013}
I20260320 15:27:47.250742 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 6d9b0f1936f54c699211c5ee9709f9ed (table default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]) successfully deleted
I20260320 15:27:47.258513 30687 raft_consensus.cc:493] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.259047 30687 raft_consensus.cc:515] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.259269 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 309) took 3241 ms. Trace:
I20260320 15:27:47.259414 29678 rpcz_store.cc:276] 0320 15:27:44.017840 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.018951 (+ 1111us) service_pool.cc:225] Handling call
0320 15:27:47.259018 (+3240067us) ts_tablet_manager.cc:1151] Deleting tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec
0320 15:27:47.259035 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.259219 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3239848}
W20260320 15:27:47.259968 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec already in progress: creating tablet
I20260320 15:27:47.260035 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 310) took 3231 ms. Trace:
I20260320 15:27:47.260124 29678 rpcz_store.cc:276] 0320 15:27:44.028749 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.028825 (+ 76us) service_pool.cc:225] Handling call
0320 15:27:47.259917 (+3231092us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:47.259927 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.259998 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3230860}
W20260320 15:27:47.260550 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fc009401bb3d440f9d36e080efef1204 because the tablet was not found. No further retry: Not found: Tablet not found: fc009401bb3d440f9d36e080efef1204
I20260320 15:27:47.260596 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 311) took 3220 ms. Trace:
I20260320 15:27:47.260672 29678 rpcz_store.cc:276] 0320 15:27:44.040369 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.040512 (+ 143us) service_pool.cc:225] Handling call
0320 15:27:47.260480 (+3219968us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:47.260489 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.260555 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3219707}
W20260320 15:27:47.261001 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bdd8f5ccbd35418684fd27459c7a6b9f because the tablet was not found. No further retry: Not found: Tablet not found: bdd8f5ccbd35418684fd27459c7a6b9f
I20260320 15:27:47.261313 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:47.261968 28382 leader_election.cc:343] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.263052 30687 leader_election.cc:290] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.265414 29678 tablet_replica.cc:333] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.267236 29678 raft_consensus.cc:2243] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.267665 29678 raft_consensus.cc:2272] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.268265 29678 ts_tablet_manager.cc:1916] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.275805 29678 ts_tablet_manager.cc:1929] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.275990 29678 log.cc:1199] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:47.276494 29678 ts_tablet_manager.cc:1950] T 8241a342054240d2ac62b729cc9c9a62 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.277387 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 314) took 3236 ms. Trace:
I20260320 15:27:47.277505 29678 rpcz_store.cc:276] 0320 15:27:44.040678 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.040733 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:47.265313 (+3224580us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:47.265329 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.271996 (+ 6667us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.275649 (+ 3653us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.277340 (+ 1691us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3224398}
I20260320 15:27:47.277679 30651 consensus_queue.cc:1048] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.278012 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8241a342054240d2ac62b729cc9c9a62 (table default.Table_f5cf94af44a94088b01b125cc76ec506 [id=d3f0c922b394404b9c5e7bc019f1368b]) successfully deleted
I20260320 15:27:47.281129 28733 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.281327 28733 ts_tablet_manager.cc:1403] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.324s user 0.004s sys 0.000s
I20260320 15:27:47.282298 28733 raft_consensus.cc:359] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.282528 28733 raft_consensus.cc:385] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.282589 28733 raft_consensus.cc:740] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.282821 28733 consensus_queue.cc:260] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:47.283632 28383 leader_election.cc:343] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.283799 28383 leader_election.cc:304] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.287333 30687 raft_consensus.cc:2749] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.291895 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
W20260320 15:27:47.294689 28383 leader_election.cc:343] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.296904 29640 ts_tablet_manager.cc:1929] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.299219 28383 leader_election.cc:304] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.299907 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 312) took 3259 ms. Trace:
I20260320 15:27:47.300034 29678 rpcz_store.cc:276] 0320 15:27:44.040463 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.040816 (+ 353us) service_pool.cc:225] Handling call
0320 15:27:47.299688 (+3258872us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:47.299703 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.299848 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3258753}
W20260320 15:27:47.300555 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 07af28f257d745a1bb4173aad09045d9 because the tablet was not found. No further retry: Not found: Tablet not found: 07af28f257d745a1bb4173aad09045d9
I20260320 15:27:47.300830 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 313) took 3260 ms. Trace:
I20260320 15:27:47.300930 29678 rpcz_store.cc:276] 0320 15:27:44.040555 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.040951 (+ 396us) service_pool.cc:225] Handling call
0320 15:27:47.300288 (+3259337us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:47.300299 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.300794 (+ 495us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3259220}
W20260320 15:27:47.301342 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 20a7164db1e0447fb424d8e46b747f52 because the tablet was not found. No further retry: Not found: Tablet not found: 20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:47.301604 29678 tablet_replica.cc:333] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.301841 29678 raft_consensus.cc:2243] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.302062 29678 raft_consensus.cc:2272] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.314564 28736 tablet_bootstrap.cc:492] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.314774 28736 ts_tablet_manager.cc:1403] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.413s user 0.030s sys 0.046s
I20260320 15:27:47.315712 28736 raft_consensus.cc:359] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.315914 28736 raft_consensus.cc:385] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.315986 28736 raft_consensus.cc:740] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.316246 28736 consensus_queue.cc:260] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.316787 28736 ts_tablet_manager.cc:1434] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.320366 30687 raft_consensus.cc:2749] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.320860 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
W20260320 15:27:47.320955 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 76379bf0fa554820be020f647582a72b consensus state has no leader
I20260320 15:27:47.321575 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:47.322484 28736 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.339658 28734 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.339818 28734 ts_tablet_manager.cc:1403] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.123s user 0.002s sys 0.029s
W20260320 15:27:47.340057 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 45383.2us
W20260320 15:27:47.340343 28383 leader_election.cc:343] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.340521 28383 leader_election.cc:304] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.340569 28734 raft_consensus.cc:359] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.340731 28734 raft_consensus.cc:385] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.340790 28734 raft_consensus.cc:740] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.340955 30687 raft_consensus.cc:2749] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.342682 29678 ts_tablet_manager.cc:1916] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.347679 28734 consensus_queue.cc:260] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.350364 29678 ts_tablet_manager.cc:1929] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:47.350555 29678 log.cc:1199] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:47.351033 29678 ts_tablet_manager.cc:1950] T 87a3d5e4ff8f42409482af49c636be0c P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.351711 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:47.351915 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 319) took 2448 ms. Trace:
I20260320 15:27:47.352049 29678 rpcz_store.cc:276] 0320 15:27:44.903315 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.903691 (+ 376us) service_pool.cc:225] Handling call
0320 15:27:47.301529 (+2397838us) ts_tablet_manager.cc:1151] Deleting tablet 87a3d5e4ff8f42409482af49c636be0c
0320 15:27:47.301541 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.342321 (+ 40780us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x60d000013028. stack: 00007f7f40429076 00007f7f40428e30 00007f7f3f3e33ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007f7f4019d9ca 00007f7f40198f76 00007f7f40290735 00007f7f40290bfd 00007f7f402957d1 00007f7f40295710 00007f7f4028fd82 000000000097fcf6 000000000097fbd1 0000000000a8fa01
0320 15:27:47.346613 (+ 4292us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.350229 (+ 3616us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.351863 (+ 1634us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":90672384,"tablet-delete.queue_time_us":2395911}
I20260320 15:27:47.352674 29640 log.cc:1199] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/aafd1e123cd043e7828140d2443e047f
I20260320 15:27:47.353272 29640 ts_tablet_manager.cc:1950] T aafd1e123cd043e7828140d2443e047f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:47.352551 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]) successfully deleted
I20260320 15:27:47.354046 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:47.336745 30655 raft_consensus.cc:493] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:47.354532 30655 raft_consensus.cc:515] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.355155 30655 leader_election.cc:290] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:47.337158 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 53912.8us
W20260320 15:27:47.356488 28524 leader_election.cc:343] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:47.358785 28524 leader_election.cc:343] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:47.359084 28524 consensus_peers.cc:597] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bd3032950254e6f9db8da35dbb1acc5. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:47.359747 28734 ts_tablet_manager.cc:1434] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260320 15:27:47.359812 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 318) took 2456 ms. Trace:
I20260320 15:27:47.360075 28734 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.359953 29678 rpcz_store.cc:276] 0320 15:27:44.903185 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.903578 (+ 393us) service_pool.cc:225] Handling call
0320 15:27:47.352388 (+2448810us) ts_tablet_manager.cc:1151] Deleting tablet e2243e6811744b7cba99a487f8e8bb83
0320 15:27:47.352401 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.359764 (+ 7363us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2447016}
I20260320 15:27:47.360764 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.360963 28447 raft_consensus.cc:2468] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:47.362212 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.362432 29678 tablet_replica.cc:333] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.362459 28296 raft_consensus.cc:2468] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:47.362679 29678 raft_consensus.cc:2243] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.362871 29678 raft_consensus.cc:2272] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.363473 29678 ts_tablet_manager.cc:1916] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.376192 28736 tablet_bootstrap.cc:654] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.335851 28733 ts_tablet_manager.cc:1434] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.054s user 0.002s sys 0.001s
I20260320 15:27:47.384732 28733 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.387126 28734 tablet_bootstrap.cc:654] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.388845 30687 raft_consensus.cc:493] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.389142 30687 raft_consensus.cc:515] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.389780 28736 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.389942 28736 ts_tablet_manager.cc:1403] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.068s user 0.005s sys 0.004s
I20260320 15:27:47.390753 28736 raft_consensus.cc:359] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.390925 28736 raft_consensus.cc:385] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.390978 28736 raft_consensus.cc:740] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.391047 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.391170 28736 consensus_queue.cc:260] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.391741 28736 ts_tablet_manager.cc:1434] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260320 15:27:47.391829 28382 leader_election.cc:343] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.392552 30687 leader_election.cc:290] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.394331 28736 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:47.361492 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e2243e6811744b7cba99a487f8e8bb83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2243e6811744b7cba99a487f8e8bb83 already in progress: creating tablet
I20260320 15:27:47.396023 28736 tablet_bootstrap.cc:654] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.397593 29678 ts_tablet_manager.cc:1929] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.397725 29678 log.cc:1199] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8d92e17cea044541968356ae993be1e7
I20260320 15:27:47.398142 29678 ts_tablet_manager.cc:1950] T 8d92e17cea044541968356ae993be1e7 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.398962 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 322) took 1520 ms. Trace:
I20260320 15:27:47.399049 29678 rpcz_store.cc:276] 0320 15:27:45.878710 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.878887 (+ 177us) service_pool.cc:225] Handling call
0320 15:27:47.362349 (+1483462us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e17cea044541968356ae993be1e7
0320 15:27:47.362361 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.367080 (+ 4719us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.397441 (+ 30361us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.398941 (+ 1500us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1483034}
I20260320 15:27:47.399426 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 323) took 1494 ms. Trace:
I20260320 15:27:47.399494 29678 rpcz_store.cc:276] 0320 15:27:45.904871 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.904930 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:47.399289 (+1494359us) ts_tablet_manager.cc:1151] Deleting tablet 7f94140a5e8b4a06890bca86456d64a6
0320 15:27:47.399300 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.399410 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1491106}
I20260320 15:27:47.399812 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 324) took 1486 ms. Trace:
I20260320 15:27:47.399880 29678 rpcz_store.cc:276] 0320 15:27:45.925603 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.925806 (+ 203us) service_pool.cc:225] Handling call
0320 15:27:47.399733 (+1473927us) ts_tablet_manager.cc:1151] Deleting tablet 6771bd6a3f0e47d99a0191eae25cb2d2
0320 15:27:47.399740 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.399799 (+ 59us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1457331}
I20260320 15:27:47.400149 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 326) took 1453 ms. Trace:
I20260320 15:27:47.400208 29678 rpcz_store.cc:276] 0320 15:27:45.946640 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.946704 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:47.400057 (+1453353us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:47.400064 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.400136 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1453131}
I20260320 15:27:47.403726 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:47.405378 30636 consensus_queue.cc:1048] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.407876 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 325) took 1464 ms. Trace:
I20260320 15:27:47.408046 29678 rpcz_store.cc:276] 0320 15:27:45.943836 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.943941 (+ 105us) service_pool.cc:225] Handling call
0320 15:27:47.400418 (+1456477us) ts_tablet_manager.cc:1151] Deleting tablet ae3422cf33324ab89a2b44e27cae0eb2
0320 15:27:47.407594 (+ 7176us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.407837 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1447740}
I20260320 15:27:47.408818 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 327) took 1450 ms. Trace:
I20260320 15:27:47.408929 29678 rpcz_store.cc:276] 0320 15:27:45.958063 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.959812 (+ 1749us) service_pool.cc:225] Handling call
0320 15:27:47.408644 (+1448832us) ts_tablet_manager.cc:1151] Deleting tablet a185f84c24ea402684aa1cc3628d4f11
0320 15:27:47.408662 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.408798 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1443176}
I20260320 15:27:47.409340 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 329) took 1425 ms. Trace:
I20260320 15:27:47.409430 29678 rpcz_store.cc:276] 0320 15:27:45.984454 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.984537 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:47.409222 (+1424685us) ts_tablet_manager.cc:1151] Deleting tablet f0a31d385b81456fb46ea52083874fcf
0320 15:27:47.409235 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.409322 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1424449}
I20260320 15:27:47.409760 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 330) took 1424 ms. Trace:
I20260320 15:27:47.409920 29678 rpcz_store.cc:276] 0320 15:27:45.985115 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.985215 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:47.409678 (+1424463us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:47.409685 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.409745 (+ 60us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1424294}
I20260320 15:27:47.410406 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 331) took 1424 ms. Trace:
I20260320 15:27:47.411129 28736 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.411303 28736 ts_tablet_manager.cc:1403] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:47.410488 29678 rpcz_store.cc:276] 0320 15:27:45.985655 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.985753 (+ 98us) service_pool.cc:225] Handling call
0320 15:27:47.410306 (+1424553us) ts_tablet_manager.cc:1151] Deleting tablet 07ff84dedc074dabb4e8e2d8460066e7
0320 15:27:47.410317 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.410387 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1424312}
I20260320 15:27:47.414232 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 328) took 1430 ms. Trace:
I20260320 15:27:47.414347 29678 rpcz_store.cc:276] 0320 15:27:45.983977 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.985817 (+ 1840us) service_pool.cc:225] Handling call
0320 15:27:47.414073 (+1428256us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:47.414089 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.414213 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1412486}
I20260320 15:27:47.414633 29678 tablet_replica.cc:333] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.414844 29678 raft_consensus.cc:2243] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.415287 29678 raft_consensus.cc:2272] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.416208 29678 ts_tablet_manager.cc:1916] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.416678 28734 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.416814 28734 ts_tablet_manager.cc:1403] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.002s
I20260320 15:27:47.417515 28734 raft_consensus.cc:359] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.417690 28734 raft_consensus.cc:385] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.417735 28734 raft_consensus.cc:740] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.417959 28734 consensus_queue.cc:260] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.418365 28734 ts_tablet_manager.cc:1434] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260320 15:27:47.418493 30466 raft_consensus.cc:493] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.418653 28734 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.418627 30466 raft_consensus.cc:515] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.419703 30466 leader_election.cc:290] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:47.421453 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.421722 28447 raft_consensus.cc:2393] T 78288f7aea204870af75060476760ee5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:47.428056 28736 raft_consensus.cc:359] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.428309 28736 raft_consensus.cc:385] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.428367 28736 raft_consensus.cc:740] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.428581 28736 consensus_queue.cc:260] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.429286 28736 ts_tablet_manager.cc:1434] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.018s user 0.001s sys 0.000s
I20260320 15:27:47.429690 28736 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.441224 29678 ts_tablet_manager.cc:1929] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.441382 29678 log.cc:1199] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:47.441799 29678 ts_tablet_manager.cc:1950] T fcff253b9cce45b4ad11dc38397afee2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.449266 28734 tablet_bootstrap.cc:654] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.454964 30456 raft_consensus.cc:493] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.455191 30456 raft_consensus.cc:515] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.455657 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 335) took 1426 ms. Trace:
I20260320 15:27:47.455770 29678 rpcz_store.cc:276] 0320 15:27:46.030405 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.030795 (+ 390us) service_pool.cc:225] Handling call
0320 15:27:47.414578 (+1383783us) ts_tablet_manager.cc:1151] Deleting tablet fcff253b9cce45b4ad11dc38397afee2
0320 15:27:47.414588 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.434995 (+ 20407us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.441083 (+ 6088us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.455631 (+ 14548us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1383520}
I20260320 15:27:47.456255 30456 leader_election.cc:290] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.457497 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:47.458585 28382 leader_election.cc:343] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.460611 29678 tablet_replica.cc:333] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.460841 29678 raft_consensus.cc:2243] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.461026 29678 raft_consensus.cc:2272] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.461169 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 180) took 11688 ms. Trace:
I20260320 15:27:47.461360 29640 rpcz_store.cc:276] 0320 15:27:35.772801 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.772849 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:46.888273 (+11115424us) ts_tablet_manager.cc:1151] Deleting tablet aafd1e123cd043e7828140d2443e047f
0320 15:27:46.888286 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.189627 (+301341us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.296766 (+107139us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.461076 (+164310us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11115255}
I20260320 15:27:47.461617 29678 ts_tablet_manager.cc:1916] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.462898 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 181) took 11689 ms. Trace:
I20260320 15:27:47.463084 29640 rpcz_store.cc:276] 0320 15:27:35.773651 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.773707 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:47.462770 (+11689063us) ts_tablet_manager.cc:1151] Deleting tablet b8f3ee3c611d41b483bd35f868042d05
0320 15:27:47.462785 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.462875 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11688903}
I20260320 15:27:47.469182 30713 tablet.cc:1723] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:47.462240 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet aafd1e123cd043e7828140d2443e047f (table default.Table_134b3b5c79e344f2bf710b12615479be [id=5186e4c77f9f4f59814887534793f0f0]) successfully deleted
I20260320 15:27:47.471203 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:47.472396 28736 tablet_bootstrap.cc:654] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.476226 30723 tablet.cc:1723] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:47.480659 29678 ts_tablet_manager.cc:1929] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.480823 29678 log.cc:1199] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:47.481176 29678 ts_tablet_manager.cc:1950] T cc97a5f2bd964a05bb7b2427226debdd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.485519 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 336) took 1452 ms. Trace:
I20260320 15:27:47.485669 29678 rpcz_store.cc:276] 0320 15:27:46.033893 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.034566 (+ 673us) service_pool.cc:225] Handling call
0320 15:27:47.460546 (+1425980us) ts_tablet_manager.cc:1151] Deleting tablet cc97a5f2bd964a05bb7b2427226debdd
0320 15:27:47.460560 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.476424 (+ 15864us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.480527 (+ 4103us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.485490 (+ 4963us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1425734}
I20260320 15:27:47.487828 29643 ts_tablet_manager.cc:1929] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.488095 29643 log.cc:1199] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/6d9b0f1936f54c699211c5ee9709f9ed
I20260320 15:27:47.488477 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:47.488715 29643 ts_tablet_manager.cc:1950] T 6d9b0f1936f54c699211c5ee9709f9ed P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:47.489136 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 320) took 3594 ms. Trace:
I20260320 15:27:47.489342 29643 rpcz_store.cc:276] 0320 15:27:43.897085 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.899414 (+ 2329us) service_pool.cc:225] Handling call
0320 15:27:47.143342 (+3243928us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:47.167620 (+ 24278us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.298357 (+130737us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.487578 (+189221us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.489114 (+ 1536us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":30564,"tablet-delete.queue_time_us":3243181}
I20260320 15:27:47.490396 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 321) took 3588 ms. Trace:
I20260320 15:27:47.490068 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 6d9b0f1936f54c699211c5ee9709f9ed (table default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]) successfully deleted
I20260320 15:27:47.493306 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 182) took 11719 ms. Trace:
W20260320 15:27:47.493403 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b8f3ee3c611d41b483bd35f868042d05 because the tablet was not found. No further retry: Not found: Tablet not found: b8f3ee3c611d41b483bd35f868042d05
I20260320 15:27:47.493474 29640 rpcz_store.cc:276] 0320 15:27:35.773782 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.773835 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:47.493183 (+11719348us) ts_tablet_manager.cc:1151] Deleting tablet 89425da569794a8fa972817023bf853d
0320 15:27:47.493196 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.493288 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11719165}
I20260320 15:27:47.466113 28524 leader_election.cc:304] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:47.493980 30655 raft_consensus.cc:2804] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:47.494096 30655 raft_consensus.cc:697] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:47.494287 28524 leader_election.cc:304] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.494334 30655 consensus_queue.cc:237] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.494705 30630 raft_consensus.cc:2804] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:47.494781 30630 raft_consensus.cc:493] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:47.494838 30630 raft_consensus.cc:3060] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:47.495136 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 89425da569794a8fa972817023bf853d because the tablet was not found. No further retry: Not found: Tablet not found: 89425da569794a8fa972817023bf853d
I20260320 15:27:47.490972 28733 tablet_bootstrap.cc:700] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Time spent opening tablet: real 0.105s user 0.002s sys 0.000s
I20260320 15:27:47.495923 28733 tablet_bootstrap.cc:654] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.495461 30629 consensus_queue.cc:1048] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260320 15:27:47.491882 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:47.496690 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:47.500129 28736 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.500313 28736 ts_tablet_manager.cc:1403] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.002s
I20260320 15:27:47.501016 28736 raft_consensus.cc:359] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.501200 28736 raft_consensus.cc:385] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.501248 28736 raft_consensus.cc:740] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.501499 28736 consensus_queue.cc:260] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.502060 28736 ts_tablet_manager.cc:1434] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:47.502369 28736 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.502380 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:47.502746 28906 catalog_manager.cc:5671] T 9b838187c21448b6909ad183f9824d2b P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:47.503926 28736 tablet_bootstrap.cc:654] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.504281 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:47.505820 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 337) took 1469 ms. Trace:
I20260320 15:27:47.505931 29678 rpcz_store.cc:276] 0320 15:27:46.037393 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.038071 (+ 678us) service_pool.cc:225] Handling call
0320 15:27:47.485979 (+1447908us) ts_tablet_manager.cc:1151] Deleting tablet 4190329df1b644c392ff8cbace9aae7a
0320 15:27:47.485991 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.505799 (+ 19808us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1447669}
I20260320 15:27:47.506341 29678 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46760 (request call id 338) took 1466 ms. Trace:
I20260320 15:27:47.506420 29678 rpcz_store.cc:276] 0320 15:27:46.041941 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.042000 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:47.506216 (+1464216us) ts_tablet_manager.cc:1151] Deleting tablet d04ab44039124f36ac60601098586b4e
0320 15:27:47.506228 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.506327 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1463964}
I20260320 15:27:47.506822 29678 tablet_replica.cc:333] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.507015 29678 raft_consensus.cc:2243] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.507184 29678 raft_consensus.cc:2272] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.507812 29678 ts_tablet_manager.cc:1916] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.511711 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 183) took 11737 ms. Trace:
I20260320 15:27:47.511909 29640 rpcz_store.cc:276] 0320 15:27:35.774385 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.774597 (+ 212us) service_pool.cc:225] Handling call
0320 15:27:47.493737 (+11719140us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:47.493748 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.511671 (+ 17923us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11718942}
I20260320 15:27:47.513240 29640 tablet_replica.cc:333] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:47.513705 29640 raft_consensus.cc:2243] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.513974 29640 raft_consensus.cc:2272] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.514912 29640 ts_tablet_manager.cc:1916] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.516207 28736 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.516393 28736 ts_tablet_manager.cc:1403] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260320 15:27:47.517171 28736 raft_consensus.cc:359] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.517376 28736 raft_consensus.cc:385] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.517437 28736 raft_consensus.cc:740] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.517691 28736 consensus_queue.cc:260] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.518206 28736 ts_tablet_manager.cc:1434] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.518513 28736 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.520051 28736 tablet_bootstrap.cc:654] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.526131 30630 raft_consensus.cc:515] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.527179 30714 tablet.cc:1723] T f59a3c9fa17a4babaa9a18f18d51f193 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:47.528805 30630 leader_election.cc:290] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.529733 29678 ts_tablet_manager.cc:1929] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.529857 29678 log.cc:1199] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:47.530234 29678 ts_tablet_manager.cc:1950] T 90cd18e2ccbf48c8b804329bd3abf847 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.531426 29678 tablet_replica.cc:333] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.536734 29678 raft_consensus.cc:2243] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.537020 29678 pending_rounds.cc:70] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:47.537087 29678 pending_rounds.cc:77] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387830510829568 op_type: NO_OP noop_request { }
I20260320 15:27:47.537204 29678 raft_consensus.cc:2889] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:47.537313 29678 raft_consensus.cc:2272] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.538043 29678 ts_tablet_manager.cc:1916] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.538842 30622 raft_consensus.cc:493] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.538990 30622 raft_consensus.cc:515] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:47.512814 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because the tablet was not found. No further retry: Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
I20260320 15:27:47.539681 30622 leader_election.cc:290] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:47.539768 28733 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.539755 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:47.539898 28733 ts_tablet_manager.cc:1403] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.155s user 0.005s sys 0.000s
I20260320 15:27:47.540246 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:47.540711 28733 raft_consensus.cc:359] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.540885 28733 raft_consensus.cc:385] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.540835 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:47.540938 28733 raft_consensus.cc:740] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.541127 28906 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:47.541123 28733 consensus_queue.cc:260] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.541607 28733 ts_tablet_manager.cc:1434] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.541882 28733 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:47.543030 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 47663.1us
I20260320 15:27:47.543882 28736 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.544021 28736 ts_tablet_manager.cc:1403] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
W20260320 15:27:47.544297 28524 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 54266dd376164673bffe829dd515b83d
I20260320 15:27:47.544657 28736 raft_consensus.cc:359] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.544817 28736 raft_consensus.cc:385] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.544868 28736 raft_consensus.cc:740] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.545077 28736 consensus_queue.cc:260] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.550683 28736 ts_tablet_manager.cc:1434] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260320 15:27:47.551011 28736 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.552423 28736 tablet_bootstrap.cc:654] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.517343 28734 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.540583 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.562374 28734 ts_tablet_manager.cc:1403] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.144s user 0.005s sys 0.005s
I20260320 15:27:47.562423 28447 raft_consensus.cc:2468] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:47.563203 28734 raft_consensus.cc:359] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.563309 29678 ts_tablet_manager.cc:1929] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:47.563395 28734 raft_consensus.cc:385] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.563436 29678 log.cc:1199] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/49654c67435a48448794879464f6cfb2
I20260320 15:27:47.563450 28734 raft_consensus.cc:740] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.563709 28734 consensus_queue.cc:260] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.563818 28240 leader_election.cc:304] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.564263 28734 ts_tablet_manager.cc:1434] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:47.564272 30622 raft_consensus.cc:2804] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:47.564343 30622 raft_consensus.cc:493] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:47.564405 30622 raft_consensus.cc:3060] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.564597 28734 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.566154 28734 tablet_bootstrap.cc:654] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.567224 30622 raft_consensus.cc:515] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.568610 29643 rpcz_store.cc:276] 0320 15:27:43.903646 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.905167 (+ 1521us) service_pool.cc:225] Handling call
0320 15:27:47.490286 (+3585119us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:47.490298 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.490378 (+ 80us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3584882}
I20260320 15:27:47.569042 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 322) took 3662 ms. Trace:
I20260320 15:27:47.569097 28736 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.569130 29643 rpcz_store.cc:276] 0320 15:27:43.906644 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.906723 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:47.568919 (+3662196us) ts_tablet_manager.cc:1151] Deleting tablet 442ca186c9d4449dade39271dec5c608
0320 15:27:47.568931 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.569023 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3662015}
I20260320 15:27:47.569235 28736 ts_tablet_manager.cc:1403] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
W20260320 15:27:47.569371 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because the tablet was not found. No further retry: Not found: Tablet not found: ecaa08ba34514c5e99a9e2d4ba9268a9
I20260320 15:27:47.569432 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 323) took 3656 ms. Trace:
I20260320 15:27:47.569491 29643 rpcz_store.cc:276] 0320 15:27:43.914377 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.914936 (+ 559us) service_pool.cc:225] Handling call
0320 15:27:47.569350 (+3654414us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:47.569361 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.569417 (+ 56us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3654111}
I20260320 15:27:47.569854 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 324) took 3634 ms. Trace:
I20260320 15:27:47.569933 28736 raft_consensus.cc:359] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.570129 28736 raft_consensus.cc:385] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.570194 28736 raft_consensus.cc:740] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:47.570329 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 442ca186c9d4449dade39271dec5c608 because the tablet was not found. No further retry: Not found: Tablet not found: 442ca186c9d4449dade39271dec5c608
W20260320 15:27:47.570657 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 5438fdc6e6944e79ac8aecd0ef774467 because the tablet was not found. No further retry: Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:47.570798 28733 tablet_bootstrap.cc:654] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.571728 28736 consensus_queue.cc:260] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.573803 30622 leader_election.cc:290] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:47.574340 28736 ts_tablet_manager.cc:1434] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:47.578778 28736 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.580021 29678 ts_tablet_manager.cc:1950] T 49654c67435a48448794879464f6cfb2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.581050 28736 tablet_bootstrap.cc:654] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.581174 29678 tablet_replica.cc:333] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.581393 29678 raft_consensus.cc:2243] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:47.581707 29678 raft_consensus.cc:2272] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.582271 29678 ts_tablet_manager.cc:1916] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.583628 30629 raft_consensus.cc:493] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.583803 30629 raft_consensus.cc:515] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.583894 30725 raft_consensus.cc:493] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:47.584060 30725 raft_consensus.cc:515] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:47.584395 30629 leader_election.cc:290] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.584753 30725 leader_election.cc:290] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.573776 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:47.591606 29643 rpcz_store.cc:276] 0320 15:27:43.935378 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.935433 (+ 55us) service_pool.cc:225] Handling call
0320 15:27:47.569758 (+3634325us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:47.569766 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.569840 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3634103}
I20260320 15:27:47.591769 28445 raft_consensus.cc:3060] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:47.592433 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 796c907919df47bf9369e45a1248f96b because the tablet was not found. No further retry: Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b
I20260320 15:27:47.592955 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 325) took 3657 ms. Trace:
I20260320 15:27:47.593044 29643 rpcz_store.cc:276] 0320 15:27:43.935844 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.935885 (+ 41us) service_pool.cc:225] Handling call
0320 15:27:47.592817 (+3656932us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:47.592830 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.592932 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3656791}
W20260320 15:27:47.593750 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because the tablet was not found. No further retry: Not found: Tablet not found: 3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:47.594274 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 326) took 3657 ms. Trace:
I20260320 15:27:47.594388 29643 rpcz_store.cc:276] 0320 15:27:43.936418 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.936493 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:47.594115 (+3657622us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:47.594132 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.594230 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3657442}
W20260320 15:27:47.594995 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because the tablet was not found. No further retry: Not found: Tablet not found: 2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:47.583156 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:47.589303 28734 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.595700 28447 raft_consensus.cc:3060] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.595794 28734 ts_tablet_manager.cc:1403] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.000s
I20260320 15:27:47.596796 28734 raft_consensus.cc:359] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.597009 28734 raft_consensus.cc:385] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.597090 28734 raft_consensus.cc:740] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.597141 30651 consensus_queue.cc:1048] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.598922 28734 consensus_queue.cc:260] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.599912 28447 raft_consensus.cc:2468] T b1b6529e062440828fca2e18b129f8e3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:47.599939 30651 consensus_queue.cc:1048] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.600065 28734 ts_tablet_manager.cc:1434] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:47.600358 28734 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.600613 28445 raft_consensus.cc:2468] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:47.601121 28240 leader_election.cc:304] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.601549 30622 raft_consensus.cc:2804] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:47.601668 30622 raft_consensus.cc:697] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:47.601938 30622 consensus_queue.cc:237] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.602639 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.602731 30651 consensus_queue.cc:1048] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.603418 30651 consensus_queue.cc:1048] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.614048 28734 tablet_bootstrap.cc:654] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.623821 28736 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.624017 28736 ts_tablet_manager.cc:1403] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.000s
I20260320 15:27:47.625038 28736 raft_consensus.cc:359] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.625222 28736 raft_consensus.cc:385] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.625269 28736 raft_consensus.cc:740] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.625456 28736 consensus_queue.cc:260] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.626029 28736 ts_tablet_manager.cc:1434] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:47.626322 28736 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:47.595499 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 327) took 3658 ms. Trace:
I20260320 15:27:47.627755 29643 rpcz_store.cc:276] 0320 15:27:43.937486 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.937603 (+ 117us) service_pool.cc:225] Handling call
0320 15:27:47.595368 (+3657765us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:47.595383 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.595473 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3657563}
W20260320 15:27:47.628500 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9176e734212849aab8d99473de02cc79 because the tablet was not found. No further retry: Not found: Tablet not found: 9176e734212849aab8d99473de02cc79
I20260320 15:27:47.629029 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 328) took 3690 ms. Trace:
I20260320 15:27:47.629169 29643 rpcz_store.cc:276] 0320 15:27:43.938290 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.938381 (+ 91us) service_pool.cc:225] Handling call
0320 15:27:47.628903 (+3690522us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:47.628918 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.629005 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3690312}
W20260320 15:27:47.629768 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because the tablet was not found. No further retry: Not found: Tablet not found: 43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:47.630362 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 329) took 3688 ms. Trace:
I20260320 15:27:47.630582 29643 rpcz_store.cc:276] 0320 15:27:43.941999 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.954574 (+ 12575us) service_pool.cc:225] Handling call
0320 15:27:47.630134 (+3675560us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:47.630147 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.630248 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3675254}
W20260320 15:27:47.631161 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7feb8477ebf841a19977a434f365f537 because the tablet was not found. No further retry: Not found: Tablet not found: 7feb8477ebf841a19977a434f365f537
I20260320 15:27:47.631664 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 331) took 3666 ms. Trace:
I20260320 15:27:47.631812 29643 rpcz_store.cc:276] 0320 15:27:43.965077 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.965185 (+ 108us) service_pool.cc:225] Handling call
0320 15:27:47.631519 (+3666334us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:47.631552 (+ 33us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.631639 (+ 87us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3666072}
I20260320 15:27:47.632141 29678 ts_tablet_manager.cc:1929] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:47.632310 29678 log.cc:1199] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/b4cad900411e4c9686701aa99206dd23
W20260320 15:27:47.632349 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ef09e4afdd1f4fce9caec4dcafa2c50c because the tablet was not found. No further retry: Not found: Tablet not found: ef09e4afdd1f4fce9caec4dcafa2c50c
I20260320 15:27:47.632715 29678 ts_tablet_manager.cc:1950] T b4cad900411e4c9686701aa99206dd23 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:47.632886 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 332) took 3665 ms. Trace:
I20260320 15:27:47.633033 29643 rpcz_store.cc:276] 0320 15:27:43.967652 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.969231 (+ 1579us) service_pool.cc:225] Handling call
0320 15:27:47.632723 (+3663492us) ts_tablet_manager.cc:1151] Deleting tablet 07ff84dedc074dabb4e8e2d8460066e7
0320 15:27:47.632733 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.632858 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3663245}
W20260320 15:27:47.633608 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 07ff84dedc074dabb4e8e2d8460066e7 because the tablet was not found. No further retry: Not found: Tablet not found: 07ff84dedc074dabb4e8e2d8460066e7
I20260320 15:27:47.634122 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 333) took 3655 ms. Trace:
I20260320 15:27:47.634244 29643 rpcz_store.cc:276] 0320 15:27:43.979726 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.979917 (+ 191us) service_pool.cc:225] Handling call
0320 15:27:47.634015 (+3654098us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:47.634044 (+ 29us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.634105 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3653841}
W20260320 15:27:47.634732 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet fc009401bb3d440f9d36e080efef1204 because the tablet was not found. No further retry: Not found: Tablet not found: fc009401bb3d440f9d36e080efef1204
I20260320 15:27:47.641167 28734 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.641350 28734 ts_tablet_manager.cc:1403] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.002s
I20260320 15:27:47.642056 28734 raft_consensus.cc:359] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.642247 28734 raft_consensus.cc:385] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.642298 28734 raft_consensus.cc:740] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.642500 28734 consensus_queue.cc:260] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.642983 28734 ts_tablet_manager.cc:1434] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260320 15:27:47.650300 28872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.60:42563, user_credentials={real_user=slave}} blocked reactor thread for 56803.6us
I20260320 15:27:47.651219 30687 raft_consensus.cc:493] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.651387 30687 raft_consensus.cc:515] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.652038 30687 leader_election.cc:290] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.652952 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:47.653601 28382 leader_election.cc:343] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.654320 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8d92e17cea044541968356ae993be1e7 (table default.Table_fa221247f0074abdbafb6ec471027c6a [id=54330d66960848a5ab7eeae53e28eb35]) successfully deleted
W20260320 15:27:47.655697 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 52344us
I20260320 15:27:47.657476 30651 consensus_queue.cc:1048] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.658735 30651 consensus_queue.cc:1048] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.659132 28733 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.659346 28733 ts_tablet_manager.cc:1403] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.118s user 0.011s sys 0.015s
I20260320 15:27:47.660401 28733 raft_consensus.cc:359] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.660671 28733 raft_consensus.cc:385] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.660750 28733 raft_consensus.cc:740] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.661016 28733 consensus_queue.cc:260] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.661654 28733 ts_tablet_manager.cc:1434] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:47.661912 28524 leader_election.cc:304] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.662284 30651 raft_consensus.cc:2804] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:47.661980 28733 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.662505 30651 raft_consensus.cc:697] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:47.662873 30651 consensus_queue.cc:237] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:47.664234 28524 leader_election.cc:343] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:47.664425 28733 tablet_bootstrap.cc:654] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:47.666448 28524 leader_election.cc:343] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.667876 30622 raft_consensus.cc:493] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.668046 30622 raft_consensus.cc:515] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.668821 30622 leader_election.cc:290] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:47.666316 30655 consensus_queue.cc:1048] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260320 15:27:47.675356 29640 ts_tablet_manager.cc:1929] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:47.679569 28733 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.679728 28733 ts_tablet_manager.cc:1403] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.005s
I20260320 15:27:47.680657 28733 raft_consensus.cc:359] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.680855 28733 raft_consensus.cc:385] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.680909 28733 raft_consensus.cc:740] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.681124 28733 consensus_queue.cc:260] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.681661 28733 ts_tablet_manager.cc:1434] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.681989 28733 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.683846 28733 tablet_bootstrap.cc:654] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.675626 29640 log.cc:1199] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/197f1935099140a98c5eb801cf44046a
I20260320 15:27:47.688740 29640 ts_tablet_manager.cc:1950] T 197f1935099140a98c5eb801cf44046a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:47.695040 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 184) took 11920 ms. Trace:
W20260320 15:27:47.695995 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
W20260320 15:27:47.696333 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 because the tablet was not found. No further retry: Not found: Tablet not found: 6771bd6a3f0e47d99a0191eae25cb2d2
W20260320 15:27:47.696544 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 54266dd376164673bffe829dd515b83d because the tablet was not found. No further retry: Not found: Tablet not found: 54266dd376164673bffe829dd515b83d
W20260320 15:27:47.696750 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ae3422cf33324ab89a2b44e27cae0eb2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae3422cf33324ab89a2b44e27cae0eb2 already in progress: creating tablet
W20260320 15:27:47.696918 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet a185f84c24ea402684aa1cc3628d4f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a185f84c24ea402684aa1cc3628d4f11 already in progress: creating tablet
W20260320 15:27:47.697094 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet f0a31d385b81456fb46ea52083874fcf because the tablet was not found. No further retry: Not found: Tablet not found: f0a31d385b81456fb46ea52083874fcf
W20260320 15:27:47.697265 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 906f207dee674a4288c8c60b5d3ff9f8 because the tablet was not found. No further retry: Not found: Tablet not found: 906f207dee674a4288c8c60b5d3ff9f8
W20260320 15:27:47.697455 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 07ff84dedc074dabb4e8e2d8460066e7 because the tablet was not found. No further retry: Not found: Tablet not found: 07ff84dedc074dabb4e8e2d8460066e7
W20260320 15:27:47.697643 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 27ccc693e7704650bd77e0c4ca11862e because the tablet was not found. No further retry: Not found: Tablet not found: 27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:47.697821 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet fcff253b9cce45b4ad11dc38397afee2 (table default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]) successfully deleted
I20260320 15:27:47.698031 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet cc97a5f2bd964a05bb7b2427226debdd (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
W20260320 15:27:47.698357 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 4190329df1b644c392ff8cbace9aae7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4190329df1b644c392ff8cbace9aae7a already in progress: creating tablet
W20260320 15:27:47.698563 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d04ab44039124f36ac60601098586b4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d04ab44039124f36ac60601098586b4e already in progress: creating tablet
W20260320 15:27:47.698758 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d823ab41194542fba5540e53e9f4680b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d823ab41194542fba5540e53e9f4680b already in progress: creating tablet
I20260320 15:27:47.698930 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 90cd18e2ccbf48c8b804329bd3abf847 (table default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]) successfully deleted
W20260320 15:27:47.699138 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet db16d86664794305bde714a7ff4d5caf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db16d86664794305bde714a7ff4d5caf already in progress: creating tablet
I20260320 15:27:47.701386 28442 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.702292 28734 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.704952 28734 tablet_bootstrap.cc:654] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.706319 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 334) took 3725 ms. Trace:
I20260320 15:27:47.706554 29643 rpcz_store.cc:276] 0320 15:27:43.984584 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.020538 (+ 35954us) service_pool.cc:225] Handling call
0320 15:27:47.706182 (+3685644us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:47.706195 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.706293 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3685431}
I20260320 15:27:47.707028 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 335) took 3686 ms. Trace:
I20260320 15:27:47.714105 30466 raft_consensus.cc:493] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.714361 30466 raft_consensus.cc:515] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.715301 30466 leader_election.cc:290] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:47.715598 29643 rpcz_store.cc:276] 0320 15:27:44.020958 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.021004 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:47.706923 (+3685919us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:47.706936 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.707011 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3685754}
I20260320 15:27:47.716092 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 336) took 3694 ms. Trace:
I20260320 15:27:47.716179 29643 rpcz_store.cc:276] 0320 15:27:44.021713 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.021757 (+ 44us) service_pool.cc:225] Handling call
0320 15:27:47.715962 (+3694205us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:47.715978 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.716071 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3694058}
I20260320 15:27:47.716367 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.716568 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 337) took 3694 ms. Trace:
I20260320 15:27:47.716661 29643 rpcz_store.cc:276] 0320 15:27:44.022426 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.042513 (+ 20087us) service_pool.cc:225] Handling call
0320 15:27:47.716418 (+3673905us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:47.716432 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.716546 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3673670}
I20260320 15:27:47.716667 28444 raft_consensus.cc:2468] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:47.717018 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 338) took 3674 ms. Trace:
I20260320 15:27:47.717085 29643 rpcz_store.cc:276] 0320 15:27:44.042995 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.054019 (+ 11024us) service_pool.cc:225] Handling call
0320 15:27:47.716938 (+3662919us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:47.716949 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.717005 (+ 56us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3662698}
I20260320 15:27:47.717447 28240 leader_election.cc:304] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.717549 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 339) took 3657 ms. Trace:
I20260320 15:27:47.717625 29643 rpcz_store.cc:276] 0320 15:27:44.059945 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.060004 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:47.717356 (+3657352us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:47.717383 (+ 27us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.717530 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3657061}
I20260320 15:27:47.717958 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 340) took 3657 ms. Trace:
I20260320 15:27:47.717960 30466 raft_consensus.cc:2804] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:47.718048 30466 raft_consensus.cc:493] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:47.718118 30466 raft_consensus.cc:3060] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.718039 29643 rpcz_store.cc:276] 0320 15:27:44.060518 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.060565 (+ 47us) service_pool.cc:225] Handling call
0320 15:27:47.717853 (+3657288us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:47.717879 (+ 26us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.717940 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3657130}
I20260320 15:27:47.718405 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 341) took 3653 ms. Trace:
I20260320 15:27:47.718487 29643 rpcz_store.cc:276] 0320 15:27:44.065006 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.065291 (+ 285us) service_pool.cc:225] Handling call
0320 15:27:47.718292 (+3653001us) ts_tablet_manager.cc:1151] Deleting tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec
0320 15:27:47.718303 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.718387 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3644760}
I20260320 15:27:47.718881 29643 tablet_replica.cc:333] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:47.719081 29643 raft_consensus.cc:2243] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.719234 29643 raft_consensus.cc:2272] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.719919 28736 tablet_bootstrap.cc:654] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.721601 30466 raft_consensus.cc:515] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.723196 30687 raft_consensus.cc:493] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.723357 30687 raft_consensus.cc:515] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.723838 30466 leader_election.cc:290] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:47.724227 30687 leader_election.cc:290] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.724557 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
W20260320 15:27:47.725214 28240 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.727119 28298 raft_consensus.cc:3060] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.728603 30466 raft_consensus.cc:493] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.728855 30466 raft_consensus.cc:515] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.729975 30466 leader_election.cc:290] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:47.730113 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
W20260320 15:27:47.730775 28240 leader_election.cc:343] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.731293 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1b6529e062440828fca2e18b129f8e3" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:47.731498 28298 raft_consensus.cc:3060] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.697182 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78288f7aea204870af75060476760ee5" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.706908 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.707494 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:47.726254 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.730844 30629 consensus_queue.cc:1048] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:47.733726 28382 leader_election.cc:343] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.734517 29640 rpcz_store.cc:276] 0320 15:27:35.774540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:35.774780 (+ 240us) service_pool.cc:225] Handling call
0320 15:27:47.513158 (+11738378us) ts_tablet_manager.cc:1151] Deleting tablet 197f1935099140a98c5eb801cf44046a
0320 15:27:47.513175 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.589635 (+ 76460us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.669598 (+ 79963us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.695012 (+ 25414us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":11738231}
I20260320 15:27:47.734880 28298 raft_consensus.cc:2468] T b1b6529e062440828fca2e18b129f8e3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:47.735487 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 185) took 10741 ms. Trace:
I20260320 15:27:47.735622 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 197f1935099140a98c5eb801cf44046a (table default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]) successfully deleted
I20260320 15:27:47.735631 29640 rpcz_store.cc:276] 0320 15:27:36.993928 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:36.993991 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:47.735349 (+10741358us) ts_tablet_manager.cc:1151] Deleting tablet 7f94140a5e8b4a06890bca86456d64a6
0320 15:27:47.735364 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.735468 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10741130}
I20260320 15:27:47.735927 29640 tablet_replica.cc:333] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:47.736150 29640 raft_consensus.cc:2243] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.736366 29640 raft_consensus.cc:2272] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:47.736532 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
I20260320 15:27:47.736989 29640 ts_tablet_manager.cc:1916] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.738526 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:47.739717 28734 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.739977 28734 ts_tablet_manager.cc:1403] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.038s user 0.005s sys 0.000s
I20260320 15:27:47.740926 28734 raft_consensus.cc:359] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.748881 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.749980 28298 raft_consensus.cc:2468] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:47.751655 28733 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.751865 28733 ts_tablet_manager.cc:1403] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.015s
I20260320 15:27:47.753012 28733 raft_consensus.cc:359] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.753261 28733 raft_consensus.cc:385] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.753327 28733 raft_consensus.cc:740] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.753609 28733 consensus_queue.cc:260] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.754364 28733 ts_tablet_manager.cc:1434] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:47.754689 28733 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.763729 28293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.764294 28300 raft_consensus.cc:3060] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.763219 28733 tablet_bootstrap.cc:654] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.765852 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.767706 27944 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskc3SESu/build/asan/bin/kudu pid 29423
I20260320 15:27:47.763741 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54c75b9deab44c829f6720480e38c80b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.770859 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:47.772428 28292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.773146 28734 raft_consensus.cc:385] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.773240 28734 raft_consensus.cc:740] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.773515 28734 consensus_queue.cc:260] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.774219 28734 ts_tablet_manager.cc:1434] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.034s user 0.002s sys 0.000s
I20260320 15:27:47.774539 28734 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.762157 29643 ts_tablet_manager.cc:1916] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:47.745019 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7840b529206d482283cc550d2c5e7606 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7840b529206d482283cc550d2c5e7606 already in progress: creating tablet
W20260320 15:27:47.780604 28873 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 81241.8us
I20260320 15:27:47.780956 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 49654c67435a48448794879464f6cfb2 (table default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]) successfully deleted
W20260320 15:27:47.781396 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b16ffb28029e4b7ca37ce36efe9e0271 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16ffb28029e4b7ca37ce36efe9e0271 already in progress: creating tablet
I20260320 15:27:47.781682 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]) successfully deleted
W20260320 15:27:47.782006 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 07af28f257d745a1bb4173aad09045d9 because the tablet was not found. No further retry: Not found: Tablet not found: 07af28f257d745a1bb4173aad09045d9
W20260320 15:27:47.782298 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet bdd8f5ccbd35418684fd27459c7a6b9f because the tablet was not found. No further retry: Not found: Tablet not found: bdd8f5ccbd35418684fd27459c7a6b9f
W20260320 15:27:47.782572 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 8241a342054240d2ac62b729cc9c9a62 because the tablet was not found. No further retry: Not found: Tablet not found: 8241a342054240d2ac62b729cc9c9a62
W20260320 15:27:47.782820 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 20a7164db1e0447fb424d8e46b747f52 because the tablet was not found. No further retry: Not found: Tablet not found: 20a7164db1e0447fb424d8e46b747f52
W20260320 15:27:47.783084 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because the tablet was not found. No further retry: Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
W20260320 15:27:47.783341 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 56fa691a26114657ad768545a30dfd3c because the tablet was not found. No further retry: Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c
W20260320 15:27:47.783679 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because the tablet was not found. No further retry: Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
W20260320 15:27:47.784024 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec already in progress: creating tablet
I20260320 15:27:47.784785 29640 ts_tablet_manager.cc:1929] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:47.784937 29640 log.cc:1199] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:47.785254 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:47.785347 29640 ts_tablet_manager.cc:1950] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:47.785704 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 186) took 10717 ms. Trace:
I20260320 15:27:47.785794 29640 rpcz_store.cc:276] 0320 15:27:37.068295 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:37.068418 (+ 123us) service_pool.cc:225] Handling call
0320 15:27:47.735879 (+10667461us) ts_tablet_manager.cc:1151] Deleting tablet 5a20d7f22cfa403184e73b24d565b60a
0320 15:27:47.735889 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.763277 (+ 27388us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.784642 (+ 21365us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.785684 (+ 1042us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10667222}
I20260320 15:27:47.786350 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 5a20d7f22cfa403184e73b24d565b60a (table default.Table_12977445848d4334ab141542db268e90 [id=a010951cfbe240fea932452776c43ad6]) successfully deleted
I20260320 15:27:47.786823 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:47.787375 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:47.791025 28298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:47.792582 29640 tablet_replica.cc:333] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:47.792878 29640 raft_consensus.cc:2243] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.793066 29640 raft_consensus.cc:2272] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:47.796602 28524 leader_election.cc:343] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 54266dd376164673bffe829dd515b83d
I20260320 15:27:47.796758 28524 leader_election.cc:304] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.796785 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "008621b6a3fc40149a1c6aff8b12ef4e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.797384 30655 raft_consensus.cc:2749] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:47.798421 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:47.798936 30630 consensus_queue.cc:1048] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.799130 30395 consensus_queue.cc:1048] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:47.799275 30630 consensus_queue.cc:1048] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:47.800151 28524 consensus_peers.cc:597] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ccbf4f32636463f8e1e005a629b92cc. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:47.800658 28524 leader_election.cc:343] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:47.800766 28524 leader_election.cc:304] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:47.800925 28524 raft_consensus.cc:2686] T 5a20d7f22cfa403184e73b24d565b60a P 80636bb608a1431b955576e296ff63bb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260320 15:27:47.801659 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.802585 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:47.803370 28734 tablet_bootstrap.cc:654] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.803807 28524 leader_election.cc:304] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.804090 30655 raft_consensus.cc:2804] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:47.804157 30655 raft_consensus.cc:493] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:47.804214 30655 raft_consensus.cc:3060] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.804593 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:47.804834 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:47.805370 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:47.807039 30655 raft_consensus.cc:515] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:47.808444 28383 leader_election.cc:343] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.808548 30655 leader_election.cc:290] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.808599 28383 leader_election.cc:304] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:47.809059 28383 consensus_peers.cc:597] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87a3d5e4ff8f42409482af49c636be0c. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:47.809198 28383 leader_election.cc:343] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.809188 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:47.809281 28383 leader_election.cc:304] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.809471 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:47.809629 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:47.809798 30636 raft_consensus.cc:2749] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.810076 30636 raft_consensus.cc:2749] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.812665 28733 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.812845 28733 ts_tablet_manager.cc:1403] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.001s
I20260320 15:27:47.813903 28733 raft_consensus.cc:359] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.814116 28733 raft_consensus.cc:385] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.814177 28733 raft_consensus.cc:740] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.814222 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:47.814572 28733 consensus_queue.cc:260] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.815418 28733 ts_tablet_manager.cc:1434] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:47.815840 28733 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:47.816524 28524 leader_election.cc:343] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.817647 28733 tablet_bootstrap.cc:654] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.823490 30687 raft_consensus.cc:493] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.823707 30687 raft_consensus.cc:515] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.824396 30687 leader_election.cc:290] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.825253 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:47.826074 28382 leader_election.cc:343] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.827931 29640 ts_tablet_manager.cc:1916] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.835305 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:47.838510 30687 raft_consensus.cc:493] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.838752 30687 raft_consensus.cc:515] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.839598 30687 leader_election.cc:290] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.840149 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:47.840787 28382 leader_election.cc:343] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.842406 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:47.862109 28239 leader_election.cc:304] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.863070 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:47.864941 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:47.865216 29643 ts_tablet_manager.cc:1929] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:47.865337 29643 log.cc:1199] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:47.865777 29643 ts_tablet_manager.cc:1950] T e2243e6811744b7cba99a487f8e8bb83 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:47.866775 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 344) took 3134 ms. Trace:
I20260320 15:27:47.866890 29643 rpcz_store.cc:276] 0320 15:27:44.731922 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.731996 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:47.718832 (+2986836us) ts_tablet_manager.cc:1151] Deleting tablet e2243e6811744b7cba99a487f8e8bb83
0320 15:27:47.718841 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.806374 (+ 87533us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.865083 (+ 58709us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.866749 (+ 1666us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2986573}
I20260320 15:27:47.866858 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:47.867275 29643 tablet_replica.cc:333] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:47.867502 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet e2243e6811744b7cba99a487f8e8bb83 (table default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]) successfully deleted
I20260320 15:27:47.871956 30466 raft_consensus.cc:2749] T 78288f7aea204870af75060476760ee5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:47.872140 28239 leader_election.cc:343] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:47.872478 28239 leader_election.cc:343] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.872774 28239 leader_election.cc:304] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:47.873229 28239 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.873327 28239 leader_election.cc:304] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:47.873728 28239 leader_election.cc:343] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.873824 28239 leader_election.cc:304] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:47.875643 30622 raft_consensus.cc:2749] T 008621b6a3fc40149a1c6aff8b12ef4e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.876001 30622 raft_consensus.cc:2749] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.876324 30622 raft_consensus.cc:2749] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.879688 28736 tablet_bootstrap.cc:492] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.879913 28736 ts_tablet_manager.cc:1403] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.254s user 0.006s sys 0.005s
I20260320 15:27:47.885265 28733 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.885448 28733 ts_tablet_manager.cc:1403] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.009s
I20260320 15:27:47.886263 28733 raft_consensus.cc:359] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.886448 28733 raft_consensus.cc:385] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.886530 28733 raft_consensus.cc:740] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.886745 28733 consensus_queue.cc:260] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.887598 28733 ts_tablet_manager.cc:1434] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:47.887900 28733 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.889637 28733 tablet_bootstrap.cc:654] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.883589 28736 raft_consensus.cc:359] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.867496 29643 raft_consensus.cc:2243] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:47.872013 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 62813us
I20260320 15:27:47.896554 28383 leader_election.cc:304] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.896901 28736 raft_consensus.cc:385] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.897006 28736 raft_consensus.cc:740] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:47.897356 28383 consensus_peers.cc:597] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc97a5f2bd964a05bb7b2427226debdd. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:47.897264 28736 consensus_queue.cc:260] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.897872 28736 ts_tablet_manager.cc:1434] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.018s user 0.000s sys 0.002s
I20260320 15:27:47.898203 28736 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:47.898224 28383 leader_election.cc:343] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.898447 28383 leader_election.cc:304] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.899197 30767 raft_consensus.cc:2749] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.897365 30687 raft_consensus.cc:2749] T 54c75b9deab44c829f6720480e38c80b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.900486 28734 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:47.900614 28734 ts_tablet_manager.cc:1403] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.126s user 0.006s sys 0.000s
I20260320 15:27:47.903759 29643 raft_consensus.cc:2272] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.903733 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:47.904176 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:47.904367 29643 ts_tablet_manager.cc:1916] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.904392 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:47.905449 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:47.906032 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:47.906256 28900 catalog_manager.cc:5671] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:47.907245 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
I20260320 15:27:47.909598 29640 ts_tablet_manager.cc:1929] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:47.909739 29640 log.cc:1199] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/20eec704800e4d8b924d97556cc2700e
I20260320 15:27:47.910063 29640 ts_tablet_manager.cc:1950] T 20eec704800e4d8b924d97556cc2700e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:47.910954 30636 consensus_queue.cc:1048] T 008621b6a3fc40149a1c6aff8b12ef4e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:47.912112 28383 leader_election.cc:343] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:47.912372 28383 leader_election.cc:304] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:47.912680 30636 raft_consensus.cc:2749] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:47.915817 28734 raft_consensus.cc:359] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.916234 28734 raft_consensus.cc:385] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.916378 28734 raft_consensus.cc:740] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.916780 28734 consensus_queue.cc:260] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.917490 28734 ts_tablet_manager.cc:1434] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:47.917825 28734 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:47.924605 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 188) took 9383 ms. Trace:
I20260320 15:27:47.924865 29640 rpcz_store.cc:276] 0320 15:27:38.541409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:38.541739 (+ 330us) service_pool.cc:225] Handling call
0320 15:27:47.792487 (+9250748us) ts_tablet_manager.cc:1151] Deleting tablet 20eec704800e4d8b924d97556cc2700e
0320 15:27:47.792504 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.898376 (+105872us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.909468 (+ 11092us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.924575 (+ 15107us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4830,"tablet-delete.queue_time_us":9250547}
I20260320 15:27:47.926717 28736 tablet_bootstrap.cc:654] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.930518 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 20eec704800e4d8b924d97556cc2700e (table default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]) successfully deleted
I20260320 15:27:47.930950 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 189) took 8694 ms. Trace:
I20260320 15:27:47.931056 29640 rpcz_store.cc:276] 0320 15:27:39.236576 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.236641 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:47.930837 (+8694196us) ts_tablet_manager.cc:1151] Deleting tablet 093ae84740d5468188b4a56fa16ded9c
0320 15:27:47.930848 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.930929 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8693956}
W20260320 15:27:47.931550 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 093ae84740d5468188b4a56fa16ded9c because the tablet was not found. No further retry: Not found: Tablet not found: 093ae84740d5468188b4a56fa16ded9c
I20260320 15:27:47.931917 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 190) took 8694 ms. Trace:
I20260320 15:27:47.932003 29640 rpcz_store.cc:276] 0320 15:27:39.237403 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.237451 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:47.931825 (+8694374us) ts_tablet_manager.cc:1151] Deleting tablet 9aa2d48a54dd403c82d7f3a64db5aeab
0320 15:27:47.931835 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.931901 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8694206}
W20260320 15:27:47.932457 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9aa2d48a54dd403c82d7f3a64db5aeab because the tablet was not found. No further retry: Not found: Tablet not found: 9aa2d48a54dd403c82d7f3a64db5aeab
I20260320 15:27:47.933486 28736 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:47.933746 28736 ts_tablet_manager.cc:1403] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.004s
I20260320 15:27:47.934834 28736 raft_consensus.cc:359] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.935326 28736 raft_consensus.cc:385] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.935451 28736 raft_consensus.cc:740] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.935616 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:47.935859 28736 consensus_queue.cc:260] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.936203 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:47.936476 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:47.937091 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:47.937458 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:47.937858 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:47.938082 28900 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:47.938663 28903 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 8428019c3b074b17aec493c14cc2b911
I20260320 15:27:47.939131 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 4cfad5cf843a437e8e15c7236d1dc5fe
I20260320 15:27:47.940102 28901 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 161) took 1530 ms. Trace:
W20260320 15:27:47.939131 28889 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55bd68c6e4364a038a815d1fc05ad25e consensus state has no leader
I20260320 15:27:47.941026 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:47.943799 28734 tablet_bootstrap.cc:654] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:47.943799 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:47.939179 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 28c453eaf49c4171b0f93d3b26690d83
W20260320 15:27:47.938786 28904 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e consensus state has no leader
I20260320 15:27:47.945060 29792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260320 15:27:47.944782 28901 rpcz_store.cc:276] 0320 15:27:46.409779 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.618138 (+208359us) service_pool.cc:225] Handling call
0320 15:27:46.636340 (+ 18202us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.636510 (+ 170us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:46.636600 (+ 90us) catalog_manager.cc:2824] Locking tablets
0320 15:27:46.636770 (+ 170us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:46.665360 (+ 28590us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:46.665464 (+ 104us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:47.938857 (+1273393us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:47.940023 (+ 1166us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.637673 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.637844 (+ 171us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.637854 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:46.637858 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:46.637981 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.637994 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.637999 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.638143 (+ 144us) tablet.cc:700] Row locks acquired
0320 15:27:46.638147 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:46.638194 (+ 47us) write_op.cc:270] Start()
0320 15:27:46.638230 (+ 36us) write_op.cc:276] Timestamp: P: 1774020466638144 usec, L: 0
0320 15:27:46.638234 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.638427 (+ 193us) log.cc:844] Serialized 759 byte log entry
0320 15:27:46.658084 (+ 19657us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.658718 (+ 634us) write_op.cc:301] APPLY: starting
0320 15:27:46.658732 (+ 14us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.658768 (+ 36us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.658799 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.658857 (+ 58us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.658883 (+ 26us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:46.658889 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:46.658944 (+ 55us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.658985 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.659001 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:46.659006 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.659072 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.659076 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":597,"apply.run_cpu_time_us":347,"apply.run_wall_time_us":347,"num_ops":2,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":834,"prepare.run_wall_time_us":835,"replication_time_us":19831}]]}
W20260320 15:27:47.945891 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 103) took 1191 ms (client timeout 1000 ms). Trace:
I20260320 15:27:47.947839 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:47.948105 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 191) took 8710 ms. Trace:
I20260320 15:27:47.948215 29640 rpcz_store.cc:276] 0320 15:27:39.237987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.238030 (+ 43us) service_pool.cc:225] Handling call
0320 15:27:47.947978 (+8709948us) ts_tablet_manager.cc:1151] Deleting tablet b82c47ae3786401d931bb0b486ca3570
0320 15:27:47.947992 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.948081 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8709794}
W20260320 15:27:47.948411 28901 rpcz_store.cc:269] 0320 15:27:46.754138 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.945810 (+1191672us) service_pool.cc:212] Skipping call since client already timed out
0320 15:27:47.945855 (+ 45us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260320 15:27:47.948742 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b82c47ae3786401d931bb0b486ca3570 because the tablet was not found. No further retry: Not found: Tablet not found: b82c47ae3786401d931bb0b486ca3570
I20260320 15:27:47.949142 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 192) took 8691 ms. Trace:
I20260320 15:27:47.949230 29640 rpcz_store.cc:276] 0320 15:27:39.257239 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.257302 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:47.949033 (+8691731us) ts_tablet_manager.cc:1151] Deleting tablet 2165371f4b5448c7a2625926082ee4c6
0320 15:27:47.949044 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.949121 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8691506}
I20260320 15:27:47.950954 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 193) took 8693 ms. Trace:
I20260320 15:27:47.951056 29640 rpcz_store.cc:276] 0320 15:27:39.258025 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.258075 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:47.950854 (+8692779us) ts_tablet_manager.cc:1151] Deleting tablet 16e31e11048b494ebe1fd4d89488b0cf
0320 15:27:47.950865 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.950936 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692605}
I20260320 15:27:47.951467 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 8428019c3b074b17aec493c14cc2b911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:47.951520 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 194) took 8693 ms. Trace:
I20260320 15:27:47.951614 29640 rpcz_store.cc:276] 0320 15:27:39.258455 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.258500 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:47.951439 (+8692939us) ts_tablet_manager.cc:1151] Deleting tablet 4becc525b59d421caf79d8cd42052118
0320 15:27:47.951447 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.951500 (+ 53us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692762}
I20260320 15:27:47.951967 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 195) took 8693 ms. Trace:
I20260320 15:27:47.952003 28889 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 160) took 1554 ms. Trace:
I20260320 15:27:47.952034 29640 rpcz_store.cc:276] 0320 15:27:39.258877 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.258925 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:47.951903 (+8692978us) ts_tablet_manager.cc:1151] Deleting tablet 6231fe30c5b34d0bb7a233abbffeb1e8
0320 15:27:47.951911 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.951954 (+ 43us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692832}
I20260320 15:27:47.952368 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 196) took 8693 ms. Trace:
I20260320 15:27:47.952430 29640 rpcz_store.cc:276] 0320 15:27:39.259305 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.259349 (+ 44us) service_pool.cc:225] Handling call
0320 15:27:47.952305 (+8692956us) ts_tablet_manager.cc:1151] Deleting tablet 7c4b53fb135741efad5d3e59a6d4eb3e
0320 15:27:47.952312 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.952356 (+ 44us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692806}
I20260320 15:27:47.952780 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 197) took 8693 ms. Trace:
I20260320 15:27:47.952135 28889 rpcz_store.cc:276] 0320 15:27:46.385841 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.616296 (+230455us) service_pool.cc:225] Handling call
0320 15:27:46.617043 (+ 747us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.617319 (+ 276us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:46.617575 (+ 256us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:46.617668 (+ 93us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:46.665526 (+ 47858us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:47.940343 (+1274817us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.618597 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.618712 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.618719 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:46.618722 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:46.618828 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.618842 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.618847 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:46.618929 (+ 82us) tablet.cc:700] Row locks acquired
0320 15:27:46.618933 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:46.618976 (+ 43us) write_op.cc:270] Start()
0320 15:27:46.619029 (+ 53us) write_op.cc:276] Timestamp: P: 1774020466618924 usec, L: 0
0320 15:27:46.619035 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.619266 (+ 231us) log.cc:844] Serialized 405 byte log entry
0320 15:27:46.658007 (+ 38741us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.658189 (+ 182us) write_op.cc:301] APPLY: starting
0320 15:27:46.658209 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.658256 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.658282 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.658333 (+ 51us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.658365 (+ 32us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:46.658372 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:46.658451 (+ 79us) log.cc:844] Serialized 23 byte log entry
0320 15:27:46.658532 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.658550 (+ 18us) write_op.cc:454] Released schema lock
0320 15:27:46.658558 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.658641 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.658645 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":129,"apply.run_cpu_time_us":442,"apply.run_wall_time_us":441,"num_ops":1,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":762,"prepare.run_wall_time_us":761,"replication_time_us":38936}]]}
I20260320 15:27:47.952839 29640 rpcz_store.cc:276] 0320 15:27:39.259730 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.259807 (+ 77us) service_pool.cc:225] Handling call
0320 15:27:47.952717 (+8692910us) ts_tablet_manager.cc:1151] Deleting tablet 796c907919df47bf9369e45a1248f96b
0320 15:27:47.952724 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.952767 (+ 43us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692724}
I20260320 15:27:47.953171 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 198) took 8661 ms. Trace:
I20260320 15:27:47.953235 29640 rpcz_store.cc:276] 0320 15:27:39.291807 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.291865 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:47.953102 (+8661237us) ts_tablet_manager.cc:1151] Deleting tablet 5438fdc6e6944e79ac8aecd0ef774467
0320 15:27:47.953109 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.953156 (+ 47us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8660960}
I20260320 15:27:47.953685 29640 tablet_replica.cc:333] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:47.953881 29640 raft_consensus.cc:2243] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.955514 28733 tablet_bootstrap.cc:492] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:47.955842 28733 ts_tablet_manager.cc:1403] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.068s user 0.004s sys 0.000s
W20260320 15:27:47.957407 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 2165371f4b5448c7a2625926082ee4c6 because the tablet was not found. No further retry: Not found: Tablet not found: 2165371f4b5448c7a2625926082ee4c6
W20260320 15:27:47.957973 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 16e31e11048b494ebe1fd4d89488b0cf because the tablet was not found. No further retry: Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
I20260320 15:27:47.958591 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:47.959628 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 4becc525b59d421caf79d8cd42052118 because the tablet was not found. No further retry: Not found: Tablet not found: 4becc525b59d421caf79d8cd42052118
W20260320 15:27:47.960012 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7c4b53fb135741efad5d3e59a6d4eb3e because the tablet was not found. No further retry: Not found: Tablet not found: 7c4b53fb135741efad5d3e59a6d4eb3e
W20260320 15:27:47.960574 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 796c907919df47bf9369e45a1248f96b because the tablet was not found. No further retry: Not found: Tablet not found: 796c907919df47bf9369e45a1248f96b
W20260320 15:27:47.960881 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 5438fdc6e6944e79ac8aecd0ef774467 because the tablet was not found. No further retry: Not found: Tablet not found: 5438fdc6e6944e79ac8aecd0ef774467
I20260320 15:27:47.962056 29643 ts_tablet_manager.cc:1929] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:47.962239 29643 log.cc:1199] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:47.962694 29643 ts_tablet_manager.cc:1950] T 87a3d5e4ff8f42409482af49c636be0c P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:47.963630 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 345) took 3226 ms. Trace:
I20260320 15:27:47.963722 29643 rpcz_store.cc:276] 0320 15:27:44.737266 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.737351 (+ 85us) service_pool.cc:225] Handling call
0320 15:27:47.867219 (+3129868us) ts_tablet_manager.cc:1151] Deleting tablet 87a3d5e4ff8f42409482af49c636be0c
0320 15:27:47.867230 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.923758 (+ 56528us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.927393 (+ 3635us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:47.963603 (+ 36210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3129625}
I20260320 15:27:47.964105 28904 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 144) took 2973 ms. Trace:
I20260320 15:27:47.964183 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 350) took 3204 ms. Trace:
I20260320 15:27:47.964262 29643 rpcz_store.cc:276] 0320 15:27:44.759308 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.759613 (+ 305us) service_pool.cc:225] Handling call
0320 15:27:47.964007 (+3204394us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:47.964018 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.964165 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3204159}
I20260320 15:27:47.964413 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 8428019c3b074b17aec493c14cc2b911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:47.964583 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 353) took 3200 ms. Trace:
I20260320 15:27:47.964651 29643 rpcz_store.cc:276] 0320 15:27:44.763720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.763780 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:47.964488 (+3200708us) ts_tablet_manager.cc:1151] Deleting tablet 71ec63fb080545de838803033f204fab
0320 15:27:47.964497 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.964567 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200547}
I20260320 15:27:47.964928 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 352) took 3201 ms. Trace:
I20260320 15:27:47.964996 29643 rpcz_store.cc:276] 0320 15:27:44.763624 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.763675 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:47.964843 (+3201168us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:47.964852 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.964914 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200803}
I20260320 15:27:47.965323 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 354) took 3201 ms. Trace:
I20260320 15:27:47.965400 29643 rpcz_store.cc:276] 0320 15:27:44.763805 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.764163 (+ 358us) service_pool.cc:225] Handling call
0320 15:27:47.965243 (+3201080us) ts_tablet_manager.cc:1151] Deleting tablet adb65832eefe4bd4b08d95aa83c7bd7b
0320 15:27:47.965251 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.965309 (+ 58us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3200895}
I20260320 15:27:47.965696 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 355) took 3201 ms. Trace:
I20260320 15:27:47.965204 28904 rpcz_store.cc:276] 0320 15:27:44.991083 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.184577 (+1193494us) service_pool.cc:225] Handling call
0320 15:27:46.184975 (+ 398us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.185197 (+ 222us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:46.185438 (+ 241us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:46.185522 (+ 84us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:46.193171 (+ 7649us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:47.964019 (+1770848us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.186743 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.186870 (+ 127us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.186878 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:46.186880 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:46.186993 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.187008 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.187013 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:46.187087 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:46.187090 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.187124 (+ 34us) write_op.cc:270] Start()
0320 15:27:46.187160 (+ 36us) write_op.cc:276] Timestamp: P: 1774020466187067 usec, L: 0
0320 15:27:46.187164 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.187356 (+ 192us) log.cc:844] Serialized 405 byte log entry
0320 15:27:46.192518 (+ 5162us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.192653 (+ 135us) write_op.cc:301] APPLY: starting
0320 15:27:46.192671 (+ 18us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.192724 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.192760 (+ 36us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.192839 (+ 79us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.192875 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:46.192884 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:46.192981 (+ 97us) log.cc:844] Serialized 23 byte log entry
0320 15:27:46.193048 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.193076 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:46.193083 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.193168 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.193173 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14118,"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":506,"apply.run_wall_time_us":506,"num_ops":1,"prepare.queue_time_us":732,"prepare.run_cpu_time_us":679,"prepare.run_wall_time_us":678,"replication_time_us":5303}]]}
I20260320 15:27:47.965757 29643 rpcz_store.cc:276] 0320 15:27:44.763906 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.764360 (+ 454us) service_pool.cc:225] Handling call
0320 15:27:47.965618 (+3201258us) ts_tablet_manager.cc:1151] Deleting tablet 906f207dee674a4288c8c60b5d3ff9f8
0320 15:27:47.965627 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.965681 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3201092}
I20260320 15:27:47.966014 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 357) took 3201 ms. Trace:
I20260320 15:27:47.966077 29643 rpcz_store.cc:276] 0320 15:27:44.764104 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.764413 (+ 309us) service_pool.cc:225] Handling call
0320 15:27:47.965926 (+3201513us) ts_tablet_manager.cc:1151] Deleting tablet 7f0242a7968541f480367b45cda5d9cc
0320 15:27:47.965934 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.966000 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3201332}
I20260320 15:27:47.966353 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 351) took 3202 ms. Trace:
I20260320 15:27:47.966444 29643 rpcz_store.cc:276] 0320 15:27:44.763475 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.764610 (+ 1135us) service_pool.cc:225] Handling call
0320 15:27:47.966280 (+3201670us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:47.966288 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.966340 (+ 52us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3201508}
I20260320 15:27:47.966732 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 348) took 3207 ms. Trace:
I20260320 15:27:47.966795 29643 rpcz_store.cc:276] 0320 15:27:44.759120 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.767595 (+ 8475us) service_pool.cc:225] Handling call
0320 15:27:47.966645 (+3199050us) ts_tablet_manager.cc:1151] Deleting tablet aafd1e123cd043e7828140d2443e047f
0320 15:27:47.966653 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.966717 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3198817}
I20260320 15:27:47.967063 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 360) took 3088 ms. Trace:
I20260320 15:27:47.967128 29643 rpcz_store.cc:276] 0320 15:27:44.879012 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.879338 (+ 326us) service_pool.cc:225] Handling call
0320 15:27:47.966981 (+3087643us) ts_tablet_manager.cc:1151] Deleting tablet 8d92e17cea044541968356ae993be1e7
0320 15:27:47.966989 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.967049 (+ 60us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3085373}
W20260320 15:27:47.967798 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 94063.6us
I20260320 15:27:47.967845 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]) successfully deleted
W20260320 15:27:47.968196 28239 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:47.969239 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 3ccbf4f32636463f8e1e005a629b92cc because the tablet was not found. No further retry: Not found: Tablet not found: 3ccbf4f32636463f8e1e005a629b92cc
W20260320 15:27:47.969796 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 71ec63fb080545de838803033f204fab because the tablet was not found. No further retry: Not found: Tablet not found: 71ec63fb080545de838803033f204fab
W20260320 15:27:47.971158 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet ad4a30476e7146afae1e77dad078136e because the tablet was not found. No further retry: Not found: Tablet not found: ad4a30476e7146afae1e77dad078136e
W20260320 15:27:47.971504 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet adb65832eefe4bd4b08d95aa83c7bd7b because the tablet was not found. No further retry: Not found: Tablet not found: adb65832eefe4bd4b08d95aa83c7bd7b
W20260320 15:27:47.972205 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 906f207dee674a4288c8c60b5d3ff9f8 because the tablet was not found. No further retry: Not found: Tablet not found: 906f207dee674a4288c8c60b5d3ff9f8
W20260320 15:27:47.972508 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7f0242a7968541f480367b45cda5d9cc because the tablet was not found. No further retry: Not found: Tablet not found: 7f0242a7968541f480367b45cda5d9cc
I20260320 15:27:47.978310 28300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:47.978593 28300 raft_consensus.cc:3060] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:47.982319 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 8dc27a4968ca4203b67dd3064fb03868 because the tablet was not found. No further retry: Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:47.982599 28300 raft_consensus.cc:2468] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:47.982712 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet aafd1e123cd043e7828140d2443e047f because the tablet was not found. No further retry: Not found: Tablet not found: aafd1e123cd043e7828140d2443e047f
I20260320 15:27:47.983048 28903 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 159) took 1580 ms. Trace:
I20260320 15:27:47.987375 29640 raft_consensus.cc:2272] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.988265 29640 ts_tablet_manager.cc:1916] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.988410 28733 raft_consensus.cc:359] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.988669 28733 raft_consensus.cc:385] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:47.988744 28733 raft_consensus.cc:740] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:47.989038 28733 consensus_queue.cc:260] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.989789 28733 ts_tablet_manager.cc:1434] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260320 15:27:47.990180 28733 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:47.992165 30725 raft_consensus.cc:493] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.992827 30719 raft_consensus.cc:493] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.992974 30719 raft_consensus.cc:515] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.992336 30725 raft_consensus.cc:515] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.993323 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:47.993669 30719 leader_election.cc:290] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:47.993811 30719 raft_consensus.cc:493] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.993844 30725 leader_election.cc:290] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.993934 30719 raft_consensus.cc:515] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.994073 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:47.994285 28447 raft_consensus.cc:2468] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:47.994524 30719 leader_election.cc:290] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:47.995343 28240 leader_election.cc:304] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:47.995812 30466 raft_consensus.cc:2804] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:47.995955 30466 raft_consensus.cc:493] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:47.996075 30466 raft_consensus.cc:3060] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:47.996583 29678 tablet_replica.cc:333] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:47.996809 29678 raft_consensus.cc:2243] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:47.997004 29678 raft_consensus.cc:2272] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:47.997648 29678 ts_tablet_manager.cc:1916] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:47.998273 30767 raft_consensus.cc:493] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:47.998422 30767 raft_consensus.cc:515] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.999045 30767 leader_election.cc:290] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:47.999307 30466 raft_consensus.cc:515] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:47.999828 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:48.000368 28382 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.000533 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.000751 28440 raft_consensus.cc:3060] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.002282 29640 ts_tablet_manager.cc:1929] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.002405 29640 log.cc:1199] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/54266dd376164673bffe829dd515b83d
I20260320 15:27:48.003453 28736 ts_tablet_manager.cc:1434] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.070s user 0.000s sys 0.002s
I20260320 15:27:48.003917 28736 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.004899 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.005172 28447 raft_consensus.cc:2468] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.002748 29640 ts_tablet_manager.cc:1950] T 54266dd376164673bffe829dd515b83d P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:48.005985 28733 tablet_bootstrap.cc:654] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.005936 28240 leader_election.cc:304] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.006249 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 199) took 8713 ms. Trace:
I20260320 15:27:48.006357 29640 rpcz_store.cc:276] 0320 15:27:39.292357 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.292402 (+ 45us) service_pool.cc:225] Handling call
0320 15:27:47.953635 (+8661233us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:47.953644 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:47.995256 (+ 41612us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.002162 (+ 6906us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.006228 (+ 4066us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33225,"tablet-delete.queue_time_us":8660949}
I20260320 15:27:48.006685 30719 raft_consensus.cc:2804] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.006807 30719 raft_consensus.cc:493] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.006925 30719 raft_consensus.cc:3060] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.008970 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 200) took 8715 ms. Trace:
I20260320 15:27:48.009078 29640 rpcz_store.cc:276] 0320 15:27:39.293211 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.293263 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:48.008823 (+8715560us) ts_tablet_manager.cc:1151] Deleting tablet b34048a722c3485490637965c60cc5dc
0320 15:27:48.008838 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.008940 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8715369}
I20260320 15:27:48.009271 29678 ts_tablet_manager.cc:1929] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.009409 29678 log.cc:1199] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/4cfad5cf843a437e8e15c7236d1dc5fe
I20260320 15:27:48.009819 29678 ts_tablet_manager.cc:1950] T 4cfad5cf843a437e8e15c7236d1dc5fe P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:48.010260 29640 tablet_replica.cc:333] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:48.010648 28736 tablet_bootstrap.cc:654] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.013643 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.013777 28734 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.013926 28734 ts_tablet_manager.cc:1403] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.005s
I20260320 15:27:48.014017 29678 tablet_replica.cc:333] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:48.014243 29678 raft_consensus.cc:2243] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.014438 29678 raft_consensus.cc:2272] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.014711 28734 raft_consensus.cc:359] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.014882 28734 raft_consensus.cc:385] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.014933 28734 raft_consensus.cc:740] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.016587 28734 consensus_queue.cc:260] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.016822 29678 ts_tablet_manager.cc:1916] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.017817 28734 ts_tablet_manager.cc:1434] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:48.020252 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 8428019c3b074b17aec493c14cc2b911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.020963 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.021974 29640 raft_consensus.cc:2243] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.021950 28440 raft_consensus.cc:2468] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.022197 29640 raft_consensus.cc:2272] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.022511 28240 leader_election.cc:304] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.022858 29640 ts_tablet_manager.cc:1916] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.023015 30623 raft_consensus.cc:2804] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.023018 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.023262 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.023590 30623 raft_consensus.cc:697] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.023900 30623 consensus_queue.cc:237] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.024111 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:48.024787 28383 leader_election.cc:343] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.024933 28383 leader_election.cc:304] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.025352 30767 raft_consensus.cc:2749] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:48.025759 28239 leader_election.cc:343] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.025873 30466 leader_election.cc:290] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:48.026134 28239 leader_election.cc:343] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:48.026685 28383 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.026822 28383 leader_election.cc:304] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.026962 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.027289 30767 raft_consensus.cc:2749] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:48.027755 28383 consensus_peers.cc:597] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cfad5cf843a437e8e15c7236d1dc5fe. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:48.015481 30719 raft_consensus.cc:515] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.030161 30719 leader_election.cc:290] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:48.030882 28892 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 158) took 1575 ms. Trace:
I20260320 15:27:48.031071 28892 rpcz_store.cc:276] 0320 15:27:46.375454 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.610687 (+235233us) service_pool.cc:225] Handling call
0320 15:27:46.611210 (+ 523us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.611384 (+ 174us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:46.611465 (+ 81us) catalog_manager.cc:2824] Locking tablets
0320 15:27:46.611655 (+ 190us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:46.654672 (+ 43017us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:46.654774 (+ 102us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:47.939117 (+1284343us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:47.950698 (+ 11581us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.612785 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.612972 (+ 187us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.612982 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:46.612987 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:46.613157 (+ 170us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.613180 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.613188 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.613344 (+ 156us) tablet.cc:700] Row locks acquired
0320 15:27:46.613347 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:46.613404 (+ 57us) write_op.cc:270] Start()
0320 15:27:46.613456 (+ 52us) write_op.cc:276] Timestamp: P: 1774020466613351 usec, L: 0
0320 15:27:46.613462 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.613743 (+ 281us) log.cc:844] Serialized 790 byte log entry
0320 15:27:46.652986 (+ 39243us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.653507 (+ 521us) write_op.cc:301] APPLY: starting
0320 15:27:46.653533 (+ 26us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.653603 (+ 70us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.653638 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.653746 (+ 108us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.653785 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:46.653791 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:46.653889 (+ 98us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.653972 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.653999 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:46.654006 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.654094 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.654097 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":442,"apply.run_cpu_time_us":578,"apply.run_wall_time_us":577,"num_ops":2,"prepare.queue_time_us":83,"prepare.run_cpu_time_us":1102,"prepare.run_wall_time_us":1102,"raft.queue_time_us":1164,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":335,"replication_time_us":39471}]]}
I20260320 15:27:48.032094 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.032351 28437 raft_consensus.cc:3060] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.023602 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.036967 28892 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_7744ce92ac4e49be9f107f055594b6b2" } modify_external_catalogs: true
I20260320 15:27:48.037267 28892 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_7744ce92ac4e49be9f107f055594b6b2" } modify_external_catalogs: true
I20260320 15:27:47.983168 28903 rpcz_store.cc:276] 0320 15:27:46.378477 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.611932 (+233455us) service_pool.cc:225] Handling call
0320 15:27:46.612584 (+ 652us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:46.612754 (+ 170us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:46.612833 (+ 79us) catalog_manager.cc:2824] Locking tablets
0320 15:27:46.612984 (+ 151us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:46.654812 (+ 41828us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:46.654866 (+ 54us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:47.938603 (+1283737us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:47.959131 (+ 20528us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.613938 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.614123 (+ 185us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.614132 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:46.614136 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:46.614287 (+ 151us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.614304 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.614312 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:46.614445 (+ 133us) tablet.cc:700] Row locks acquired
0320 15:27:46.614449 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:46.614488 (+ 39us) write_op.cc:270] Start()
0320 15:27:46.614562 (+ 74us) write_op.cc:276] Timestamp: P: 1774020466614440 usec, L: 0
0320 15:27:46.614566 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.614870 (+ 304us) log.cc:844] Serialized 759 byte log entry
0320 15:27:46.653091 (+ 38221us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.654176 (+ 1085us) write_op.cc:301] APPLY: starting
0320 15:27:46.654188 (+ 12us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.654221 (+ 33us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.654255 (+ 34us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.654333 (+ 78us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.654361 (+ 28us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:46.654367 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:46.654425 (+ 58us) log.cc:844] Serialized 29 byte log entry
0320 15:27:46.654469 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.654483 (+ 14us) write_op.cc:454] Released schema lock
0320 15:27:46.654488 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.654556 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.654560 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1042,"apply.run_cpu_time_us":372,"apply.run_wall_time_us":372,"num_ops":2,"prepare.queue_time_us":161,"prepare.run_cpu_time_us":1069,"prepare.run_wall_time_us":1532,"raft.queue_time_us":455,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":38500}]]}
I20260320 15:27:48.041294 28437 raft_consensus.cc:2468] T f6240d441c624b238f7c13eb3c5fa9fc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.042264 29640 ts_tablet_manager.cc:1929] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:48.042544 29640 log.cc:1199] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/ad4a30476e7146afae1e77dad078136e
I20260320 15:27:48.042896 29640 ts_tablet_manager.cc:1950] T ad4a30476e7146afae1e77dad078136e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:48.043412 28240 leader_election.cc:304] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.043843 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 201) took 8750 ms. Trace:
I20260320 15:27:48.043954 30719 raft_consensus.cc:2804] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.043939 29640 rpcz_store.cc:276] 0320 15:27:39.293690 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.298900 (+ 5210us) service_pool.cc:225] Handling call
0320 15:27:48.010205 (+8711305us) ts_tablet_manager.cc:1151] Deleting tablet ad4a30476e7146afae1e77dad078136e
0320 15:27:48.010215 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.026398 (+ 16183us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.042133 (+ 15735us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.043819 (+ 1686us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11440,"tablet-delete.queue_time_us":8711054}
I20260320 15:27:48.044121 30719 raft_consensus.cc:697] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.044509 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 202) took 8744 ms. Trace:
I20260320 15:27:48.044476 30719 consensus_queue.cc:237] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.044584 29640 rpcz_store.cc:276] 0320 15:27:39.299715 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.299821 (+ 106us) service_pool.cc:225] Handling call
0320 15:27:48.044398 (+8744577us) ts_tablet_manager.cc:1151] Deleting tablet 8dc27a4968ca4203b67dd3064fb03868
0320 15:27:48.044411 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.044493 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8744400}
I20260320 15:27:48.047564 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:48.048074 28239 leader_election.cc:343] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.048786 28383 leader_election.cc:304] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.049157 30767 raft_consensus.cc:2749] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.049793 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.050426 28734 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.050516 30395 raft_consensus.cc:493] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:48.050648 30395 raft_consensus.cc:515] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.051200 30395 leader_election.cc:290] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:48.052917 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 4cfad5cf843a437e8e15c7236d1dc5fe (table default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]) successfully deleted
W20260320 15:27:48.053337 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 8d92e17cea044541968356ae993be1e7 because the tablet was not found. No further retry: Not found: Tablet not found: 8d92e17cea044541968356ae993be1e7
I20260320 15:27:48.053858 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 54266dd376164673bffe829dd515b83d (table default.Table_93591206ed3f475cb1d454da2ea159a3 [id=159b136f6a17460b80f95da566dc7984]) successfully deleted
W20260320 15:27:48.054136 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b34048a722c3485490637965c60cc5dc because the tablet was not found. No further retry: Not found: Tablet not found: b34048a722c3485490637965c60cc5dc
I20260320 15:27:48.055253 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet ad4a30476e7146afae1e77dad078136e (table default.Table_30e46b102a5d409895faee96f35b2b36 [id=4fbac4a09dde4d88b82808f90c164016]) successfully deleted
W20260320 15:27:48.055513 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 8dc27a4968ca4203b67dd3064fb03868 because the tablet was not found. No further retry: Not found: Tablet not found: 8dc27a4968ca4203b67dd3064fb03868
I20260320 15:27:48.057386 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 362) took 2958 ms. Trace:
I20260320 15:27:48.057552 29643 rpcz_store.cc:276] 0320 15:27:45.098778 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.098841 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:48.054756 (+2955915us) ts_tablet_manager.cc:1151] Deleting tablet a185f84c24ea402684aa1cc3628d4f11
0320 15:27:48.057166 (+ 2410us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.057352 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2955688}
I20260320 15:27:48.062912 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_a3cd321472b34384aadb6e966f5402da" } new_table_name: "default.Table_da8719eda33f44a4a0bd0f5fe838515f" modify_external_catalogs: true
W20260320 15:27:48.069067 28239 leader_election.cc:343] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.075007 28905 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_ab12be810262486ca2ec3234763b510b [id=bf25edfb9f0e4924aadf3e85e57fab77] alter complete (version 1)
W20260320 15:27:48.075067 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 82435us
I20260320 15:27:48.076870 30629 consensus_queue.cc:1048] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.078933 28524 leader_election.cc:304] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.079283 30629 raft_consensus.cc:2804] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.079422 30629 raft_consensus.cc:697] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.079789 30629 consensus_queue.cc:237] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.086042 28892 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 6049ed1c2d84494893bfa20a9d563838
I20260320 15:27:48.064442 28734 tablet_bootstrap.cc:654] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.087671 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 6049ed1c2d84494893bfa20a9d563838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.087051 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet 6049ed1c2d84494893bfa20a9d563838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:48.087965 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet a185f84c24ea402684aa1cc3628d4f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a185f84c24ea402684aa1cc3628d4f11 already in progress: creating tablet
I20260320 15:27:48.088435 29643 tablet_replica.cc:333] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.088832 29643 raft_consensus.cc:2243] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.089197 29643 raft_consensus.cc:2272] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.090020 29643 ts_tablet_manager.cc:1916] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.090169 28892 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 162) took 1603 ms. Trace:
I20260320 15:27:48.090364 28892 rpcz_store.cc:276] 0320 15:27:46.486350 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.032027 (+1545677us) service_pool.cc:225] Handling call
0320 15:27:48.037408 (+ 5381us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:48.037588 (+ 180us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:48.037710 (+ 122us) catalog_manager.cc:2824] Locking tablets
0320 15:27:48.037932 (+ 222us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:48.061356 (+ 23424us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:48.061461 (+ 105us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:48.061504 (+ 43us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:48.090119 (+ 28615us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.040656 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.040807 (+ 151us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.040815 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:48.040817 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:48.040948 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.040966 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.040971 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:48.052223 (+ 11252us) tablet.cc:700] Row locks acquired
0320 15:27:48.052229 (+ 6us) write_op.cc:260] PREPARE: finished
0320 15:27:48.052298 (+ 69us) write_op.cc:270] Start()
0320 15:27:48.052340 (+ 42us) write_op.cc:276] Timestamp: P: 1774020468052247 usec, L: 0
0320 15:27:48.052344 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.054619 (+ 2275us) log.cc:844] Serialized 725 byte log entry
0320 15:27:48.059975 (+ 5356us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.060096 (+ 121us) write_op.cc:301] APPLY: starting
0320 15:27:48.060116 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.060167 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.060194 (+ 27us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.060271 (+ 77us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.060322 (+ 51us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:48.060329 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:48.060415 (+ 86us) log.cc:844] Serialized 29 byte log entry
0320 15:27:48.060472 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.060493 (+ 21us) write_op.cc:454] Released schema lock
0320 15:27:48.060500 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.060584 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.060588 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":66,"apply.run_cpu_time_us":477,"apply.run_wall_time_us":475,"num_ops":2,"prepare.queue_time_us":1686,"prepare.run_cpu_time_us":1114,"prepare.run_wall_time_us":14127,"raft.queue_time_us":83,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":1025,"replication_time_us":7598}]]}
I20260320 15:27:48.091881 28733 tablet_bootstrap.cc:492] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.092099 28733 ts_tablet_manager.cc:1403] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.102s user 0.003s sys 0.006s
I20260320 15:27:48.093150 28733 raft_consensus.cc:359] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.093360 28733 raft_consensus.cc:385] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.093412 28733 raft_consensus.cc:740] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:48.095608 28903 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6725124b62ad4171a2cfe072687b8c3d consensus state has no leader
I20260320 15:27:48.095718 28278 tablet_service.cc:1558] Processing DeleteTablet for tablet 6049ed1c2d84494893bfa20a9d563838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.096943 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 163) took 1540 ms. Trace:
I20260320 15:27:48.100733 29678 ts_tablet_manager.cc:1929] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.100901 29678 log.cc:1199] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/28c453eaf49c4171b0f93d3b26690d83
I20260320 15:27:48.101356 29678 ts_tablet_manager.cc:1950] T 28c453eaf49c4171b0f93d3b26690d83 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:48.104913 30655 consensus_queue.cc:1048] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.106374 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]) successfully deleted
I20260320 15:27:48.093677 28733 consensus_queue.cc:260] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.110741 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.110975 28437 raft_consensus.cc:2468] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:48.111119 28733 ts_tablet_manager.cc:1434] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260320 15:27:48.111477 28733 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:48.116864 28734 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.117020 28734 ts_tablet_manager.cc:1403] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.067s user 0.010s sys 0.010s
I20260320 15:27:48.117722 28734 raft_consensus.cc:359] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.117946 28734 raft_consensus.cc:385] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.118010 28734 raft_consensus.cc:740] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.118264 28734 consensus_queue.cc:260] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.118892 28734 ts_tablet_manager.cc:1434] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:48.119161 28734 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.119889 28892 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_dfd551c35b134e27a2424da18f1a5bd3" } new_table_name: "default.Table_1671adae9cbe4d76a53e9544d76d5497" modify_external_catalogs: true
I20260320 15:27:48.121243 28734 tablet_bootstrap.cc:654] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.123669 30655 raft_consensus.cc:493] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.123852 30655 raft_consensus.cc:515] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.124775 30655 leader_election.cc:290] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.097086 28903 rpcz_store.cc:276] 0320 15:27:46.555064 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.039626 (+1484562us) service_pool.cc:225] Handling call
0320 15:27:48.063177 (+ 23551us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:48.063429 (+ 252us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:48.063688 (+ 259us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:48.063778 (+ 90us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:48.093761 (+ 29983us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:48.095826 (+ 2065us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.067266 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.067406 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.067413 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:48.067415 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:48.067596 (+ 181us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.067609 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.067615 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:48.067689 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:48.067691 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:48.067732 (+ 41us) write_op.cc:270] Start()
0320 15:27:48.067774 (+ 42us) write_op.cc:276] Timestamp: P: 1774020468067695 usec, L: 0
0320 15:27:48.067779 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.068006 (+ 227us) log.cc:844] Serialized 405 byte log entry
0320 15:27:48.083569 (+ 15563us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.083687 (+ 118us) write_op.cc:301] APPLY: starting
0320 15:27:48.083707 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.083754 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.083787 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.083846 (+ 59us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.083881 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.083890 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:48.093232 (+ 9342us) log.cc:844] Serialized 23 byte log entry
0320 15:27:48.093457 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.093487 (+ 30us) write_op.cc:454] Released schema lock
0320 15:27:48.093498 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.093626 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.093631 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":732,"apply.run_wall_time_us":9915,"num_ops":1,"prepare.queue_time_us":2956,"prepare.run_cpu_time_us":842,"prepare.run_wall_time_us":842,"replication_time_us":15727}]]}
I20260320 15:27:48.130569 28903 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_f81d07130d8943e5bb8ba81230e2e6b7" } new_table_name: "default.Table_8ef5be02dff04c058181c86aa2de56a0" modify_external_catalogs: true
I20260320 15:27:48.138142 28736 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.138346 28736 ts_tablet_manager.cc:1403] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.135s user 0.006s sys 0.003s
I20260320 15:27:48.139075 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.139245 28736 raft_consensus.cc:359] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.139406 28736 raft_consensus.cc:385] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.139501 28736 raft_consensus.cc:740] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.139820 28736 consensus_queue.cc:260] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.140241 28736 ts_tablet_manager.cc:1434] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:48.101972 29678 tablet_replica.cc:333] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:48.142982 28736 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.145501 30767 raft_consensus.cc:493] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.145713 30767 raft_consensus.cc:515] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.146461 28736 tablet_bootstrap.cc:654] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.146652 30767 leader_election.cc:290] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.148193 28733 tablet_bootstrap.cc:654] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.149684 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e9791ac61b34fc09a8292c79f0c4032" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.150416 28734 tablet_bootstrap.cc:492] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.150581 28734 ts_tablet_manager.cc:1403] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.004s
I20260320 15:27:48.151307 28734 raft_consensus.cc:359] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.151482 28734 raft_consensus.cc:385] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.151597 28734 raft_consensus.cc:740] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.151840 28524 leader_election.cc:304] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.151863 28734 consensus_queue.cc:260] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.152099 30395 raft_consensus.cc:2804] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.152169 30395 raft_consensus.cc:493] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.152233 30395 raft_consensus.cc:3060] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.152536 28734 ts_tablet_manager.cc:1434] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:48.152974 28734 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.153159 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb7cbe0a9224412880144927ec8fa99" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.154914 30395 raft_consensus.cc:515] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.154767 28734 tablet_bootstrap.cc:654] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.155658 30395 leader_election.cc:290] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:48.156009 28524 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:48.157090 28736 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.157588 28736 ts_tablet_manager.cc:1403] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:48.158363 28736 raft_consensus.cc:359] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.158560 28736 raft_consensus.cc:385] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.158634 28736 raft_consensus.cc:740] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.158854 28736 consensus_queue.cc:260] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.159305 28736 ts_tablet_manager.cc:1434] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:48.159735 28736 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:48.160620 28382 leader_election.cc:343] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.161612 28736 tablet_bootstrap.cc:654] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.162581 29678 raft_consensus.cc:2243] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:48.162984 29678 raft_consensus.cc:2272] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.163651 29678 ts_tablet_manager.cc:1916] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.158231 30395 consensus_queue.cc:1048] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260320 15:27:48.165004 28524 leader_election.cc:343] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:48.167279 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.167490 28439 raft_consensus.cc:3060] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.169818 29640 tablet_replica.cc:333] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:48.170221 29640 raft_consensus.cc:2243] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:48.170961 29640 raft_consensus.cc:2272] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.172362 29640 ts_tablet_manager.cc:1916] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.173744 28439 raft_consensus.cc:2468] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.174412 28734 tablet_bootstrap.cc:492] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.175015 28734 ts_tablet_manager.cc:1403] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260320 15:27:48.176438 28734 raft_consensus.cc:359] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.176770 28734 raft_consensus.cc:385] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.176879 28734 raft_consensus.cc:740] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.177265 28734 consensus_queue.cc:260] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.177829 28524 leader_election.cc:343] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:48.177973 28524 leader_election.cc:304] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.178785 28734 ts_tablet_manager.cc:1434] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:48.179158 30395 raft_consensus.cc:2749] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:48.179244 28734 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.181358 28733 tablet_bootstrap.cc:492] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.184253 28733 ts_tablet_manager.cc:1403] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.073s user 0.005s sys 0.001s
W20260320 15:27:48.182410 28524 leader_election.cc:343] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.186306 29643 ts_tablet_manager.cc:1929] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.186477 29643 log.cc:1199] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:48.186712 30395 consensus_queue.cc:1048] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.187177 29643 ts_tablet_manager.cc:1950] T fcff253b9cce45b4ad11dc38397afee2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
W20260320 15:27:48.188162 28524 leader_election.cc:343] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.188380 28524 leader_election.cc:304] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.188868 30395 raft_consensus.cc:2749] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.188567 28733 raft_consensus.cc:359] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.190431 28733 raft_consensus.cc:385] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.190577 28733 raft_consensus.cc:740] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.191071 28734 tablet_bootstrap.cc:654] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.191018 28733 consensus_queue.cc:260] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.192382 28736 tablet_bootstrap.cc:492] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.192569 28736 ts_tablet_manager.cc:1403] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20260320 15:27:48.194072 28733 ts_tablet_manager.cc:1434] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260320 15:27:48.194053 28736 raft_consensus.cc:359] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.194399 28736 raft_consensus.cc:385] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.194505 28736 raft_consensus.cc:740] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.194763 28736 consensus_queue.cc:260] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.195391 28736 ts_tablet_manager.cc:1434] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260320 15:27:48.195494 30767 raft_consensus.cc:493] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.195797 28736 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.195693 30767 raft_consensus.cc:515] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.196492 30767 leader_election.cc:290] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.197345 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.197508 28524 leader_election.cc:304] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:48.198210 28382 leader_election.cc:343] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.198388 30395 raft_consensus.cc:2804] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.198694 30395 raft_consensus.cc:697] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.198854 28736 tablet_bootstrap.cc:654] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.199102 30395 consensus_queue.cc:237] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.201299 28733 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:48.206707 28736 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.206909 28736 ts_tablet_manager.cc:1403] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260320 15:27:48.208055 28736 raft_consensus.cc:359] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.208307 28736 raft_consensus.cc:385] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.208389 28736 raft_consensus.cc:740] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.208676 28736 consensus_queue.cc:260] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.209389 28736 ts_tablet_manager.cc:1434] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:48.209786 28736 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.213042 28734 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.213266 28734 ts_tablet_manager.cc:1403] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.000s
I20260320 15:27:48.214269 28734 raft_consensus.cc:359] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.214507 28734 raft_consensus.cc:385] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.214594 28734 raft_consensus.cc:740] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.214869 28734 consensus_queue.cc:260] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.215631 28734 ts_tablet_manager.cc:1434] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260320 15:27:48.216377 28734 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.218384 28734 tablet_bootstrap.cc:654] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.219157 30629 raft_consensus.cc:493] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.219314 30629 raft_consensus.cc:515] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.219748 28736 tablet_bootstrap.cc:654] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.220468 30629 leader_election.cc:290] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.221343 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.221753 28447 raft_consensus.cc:2468] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:48.221866 29678 ts_tablet_manager.cc:1929] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.222044 29678 log.cc:1199] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/8428019c3b074b17aec493c14cc2b911
I20260320 15:27:48.222561 29678 ts_tablet_manager.cc:1950] T 8428019c3b074b17aec493c14cc2b911 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:48.221510 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 366) took 2881 ms. Trace:
I20260320 15:27:48.223021 29643 rpcz_store.cc:276] 0320 15:27:45.340022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.344084 (+ 4062us) service_pool.cc:225] Handling call
0320 15:27:48.088353 (+2744269us) ts_tablet_manager.cc:1151] Deleting tablet fcff253b9cce45b4ad11dc38397afee2
0320 15:27:48.088371 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.156910 (+ 68539us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.186066 (+ 29156us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.221438 (+ 35372us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2743816}
I20260320 15:27:48.224076 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 367) took 2883 ms. Trace:
I20260320 15:27:48.226717 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:48.228708 28736 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.228878 28736 ts_tablet_manager.cc:1403] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.003s
I20260320 15:27:48.229869 28736 raft_consensus.cc:359] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.230129 28736 raft_consensus.cc:385] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.230202 28736 raft_consensus.cc:740] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.232371 28736 consensus_queue.cc:260] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.232744 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet fcff253b9cce45b4ad11dc38397afee2 (table default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]) successfully deleted
I20260320 15:27:48.233167 28736 ts_tablet_manager.cc:1434] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:48.233587 28736 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.235898 28736 tablet_bootstrap.cc:654] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.232049 28734 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.237339 28734 ts_tablet_manager.cc:1403] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.003s
I20260320 15:27:48.237195 29643 rpcz_store.cc:276] 0320 15:27:45.340162 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.344624 (+ 4462us) service_pool.cc:225] Handling call
0320 15:27:48.223919 (+2879295us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:48.223934 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.224027 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2878993}
W20260320 15:27:48.237839 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 27ccc693e7704650bd77e0c4ca11862e because the tablet was not found. No further retry: Not found: Tablet not found: 27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:48.238161 28734 raft_consensus.cc:359] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.238426 28734 raft_consensus.cc:385] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.238499 28734 raft_consensus.cc:740] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.238750 28734 consensus_queue.cc:260] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.239353 28734 ts_tablet_manager.cc:1434] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:48.239794 28734 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.241428 28524 leader_election.cc:304] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.241884 30629 raft_consensus.cc:2804] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.241937 28734 tablet_bootstrap.cc:654] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.241971 30629 raft_consensus.cc:493] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.242033 30629 raft_consensus.cc:3060] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.246009 30629 raft_consensus.cc:515] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.248416 30629 leader_election.cc:290] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.249698 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 347) took 3490 ms. Trace:
I20260320 15:27:48.249927 29643 rpcz_store.cc:276] 0320 15:27:44.758999 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.763598 (+ 4599us) service_pool.cc:225] Handling call
0320 15:27:48.249444 (+3485846us) ts_tablet_manager.cc:1151] Deleting tablet f0a31d385b81456fb46ea52083874fcf
0320 15:27:48.249462 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.249643 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2856683}
W20260320 15:27:48.251284 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet f0a31d385b81456fb46ea52083874fcf because the tablet was not found. No further retry: Not found: Tablet not found: f0a31d385b81456fb46ea52083874fcf
I20260320 15:27:48.252341 29643 tablet_replica.cc:333] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.256364 28736 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.256570 28736 ts_tablet_manager.cc:1403] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.009s
I20260320 15:27:48.257057 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb7cbe0a9224412880144927ec8fa99" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.257275 28297 raft_consensus.cc:2468] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:48.257472 28736 raft_consensus.cc:359] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.257689 28736 raft_consensus.cc:385] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.257762 28736 raft_consensus.cc:740] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.257966 28736 consensus_queue.cc:260] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.257472 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.258494 28736 ts_tablet_manager.cc:1434] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:48.258899 28736 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.260692 28736 tablet_bootstrap.cc:654] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.261368 28733 tablet_bootstrap.cc:654] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:48.264063 28524 consensus_peers.cc:597] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90cd18e2ccbf48c8b804329bd3abf847. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:48.264055 28303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:48.264384 28524 leader_election.cc:343] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:48.266142 28524 leader_election.cc:343] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:48.266608 28524 leader_election.cc:343] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.266166 29640 ts_tablet_manager.cc:1929] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.266924 29640 log.cc:1199] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:48.267345 29640 ts_tablet_manager.cc:1950] T 7a4b5a658cd84b04b50b54973c065889 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:48.269150 28562 tablet_service.cc:1511] Processing CreateTablet for tablet d38a43965b1545d29041bedd99f13619 (DEFAULT_TABLE table=default.Table_44d6e0bede5f4c87971a0e127aef1be0 [id=ddfa59d1ea4e4064a54f5e9bbe73d51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.269630 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38a43965b1545d29041bedd99f13619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.271260 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 203) took 8970 ms. Trace:
I20260320 15:27:48.271366 29640 rpcz_store.cc:276] 0320 15:27:39.300404 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.300461 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:48.169657 (+8869196us) ts_tablet_manager.cc:1151] Deleting tablet 7a4b5a658cd84b04b50b54973c065889
0320 15:27:48.169705 (+ 48us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.218454 (+ 48749us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.266032 (+ 47578us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.271229 (+ 5197us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2986,"tablet-delete.queue_time_us":8868996}
I20260320 15:27:48.271745 29640 tablet_replica.cc:333] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:48.271907 28552 tablet_service.cc:1511] Processing CreateTablet for tablet 068515f077624aed83309109eb59d077 (DEFAULT_TABLE table=default.Table_fe5422b8425b4fe9988570705b9fddcd [id=6e984ba06fc44c889a663e50b762d4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.272267 28552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068515f077624aed83309109eb59d077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.275844 28551 tablet_service.cc:1511] Processing CreateTablet for tablet e61e0f9764fe4049975b65f6221feb78 (DEFAULT_TABLE table=default.Table_3aeb90fb3c79479ea771ab8d88f6bbf4 [id=f271402e7ac84029be88aeae6beddfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.275956 29640 raft_consensus.cc:2243] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:48.276233 28551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61e0f9764fe4049975b65f6221feb78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.276317 29640 raft_consensus.cc:2272] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.276875 29640 ts_tablet_manager.cc:1916] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.253584 28734 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.279335 28734 ts_tablet_manager.cc:1403] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.006s
I20260320 15:27:48.279886 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd50f4df4b34c8d8f3460514533508a (DEFAULT_TABLE table=default.Table_74ed94505ea049559e4e388efe568e9b [id=4cadab46e6394618ada8be7ebe446f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.280282 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd50f4df4b34c8d8f3460514533508a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.280190 28734 raft_consensus.cc:359] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.280401 28734 raft_consensus.cc:385] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.263410 28383 leader_election.cc:304] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.284361 30636 raft_consensus.cc:2804] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.284461 30636 raft_consensus.cc:493] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.284523 30636 raft_consensus.cc:3060] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.285531 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.285975 30623 raft_consensus.cc:493] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.286106 30623 raft_consensus.cc:515] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.286722 30623 leader_election.cc:290] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.287261 30636 raft_consensus.cc:515] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.287734 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 7a4b5a658cd84b04b50b54973c065889 (table default.Table_a9395e2f7cc54d5eac9e394cdb342ad9 [id=5d11a272a63c4e6190471ed7eea14d61]) successfully deleted
I20260320 15:27:48.287948 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4190329df1b644c392ff8cbace9aae7a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.288111 28439 raft_consensus.cc:2468] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.288785 28278 tablet_service.cc:1511] Processing CreateTablet for tablet d38a43965b1545d29041bedd99f13619 (DEFAULT_TABLE table=default.Table_44d6e0bede5f4c87971a0e127aef1be0 [id=ddfa59d1ea4e4064a54f5e9bbe73d51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.288825 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 068515f077624aed83309109eb59d077 (DEFAULT_TABLE table=default.Table_fe5422b8425b4fe9988570705b9fddcd [id=6e984ba06fc44c889a663e50b762d4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.289139 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 8428019c3b074b17aec493c14cc2b911 (table default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]) successfully deleted
I20260320 15:27:48.289317 28278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38a43965b1545d29041bedd99f13619. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:48.289434 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6049ed1c2d84494893bfa20a9d563838 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6049ed1c2d84494893bfa20a9d563838 already in progress: creating tablet
W20260320 15:27:48.289538 28383 leader_election.cc:343] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.289644 28383 leader_election.cc:304] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.289851 28271 tablet_service.cc:1511] Processing CreateTablet for tablet e61e0f9764fe4049975b65f6221feb78 (DEFAULT_TABLE table=default.Table_3aeb90fb3c79479ea771ab8d88f6bbf4 [id=f271402e7ac84029be88aeae6beddfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.290277 28271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61e0f9764fe4049975b65f6221feb78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.290390 30767 raft_consensus.cc:2749] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.290818 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068515f077624aed83309109eb59d077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.292088 28280 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd50f4df4b34c8d8f3460514533508a (DEFAULT_TABLE table=default.Table_74ed94505ea049559e4e388efe568e9b [id=4cadab46e6394618ada8be7ebe446f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.292750 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd50f4df4b34c8d8f3460514533508a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.294173 28274 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc8e0fa469446fcac5ed9fb048cd431 (DEFAULT_TABLE table=default.Table_180c8e69093147538db5cd1b8489fa6b [id=cbff53e2a2fb46c2a2881bc0f962262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.294575 28274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc8e0fa469446fcac5ed9fb048cd431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.296028 30636 leader_election.cc:290] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.299973 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3d9cd2c074431194d805c51ba1da44 (DEFAULT_TABLE table=default.Table_228877df377e4f8bbe522f4d57092b2e [id=e95339498f7b4d1ba1e6b83c26ff39f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.300530 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d9cd2c074431194d805c51ba1da44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.303978 28277 tablet_service.cc:1511] Processing CreateTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 (DEFAULT_TABLE table=default.Table_4478d6addfd34e128e575a94d94c09f3 [id=248f8c828a0e46d7ae560e84fdef293c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.304659 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c37fc78b5d1546f69723cac88ce26ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.306841 29643 raft_consensus.cc:2243] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:48.307260 29643 raft_consensus.cc:2272] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.307914 29643 ts_tablet_manager.cc:1916] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.308023 28266 tablet_service.cc:1511] Processing CreateTablet for tablet c4c8511f0fc9412ca9754ffa9aa84c8b (DEFAULT_TABLE table=default.Table_8af7bea7b0024b3c93158cfcb8d04bbc [id=589823637776495fa71481920826e341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.308475 28266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c8511f0fc9412ca9754ffa9aa84c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.312052 28240 leader_election.cc:304] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.311862 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:48.312441 30719 raft_consensus.cc:493] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.312608 30719 raft_consensus.cc:515] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.312824 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:48.313441 30719 leader_election.cc:290] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.313642 30719 raft_consensus.cc:2804] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.313755 30719 raft_consensus.cc:493] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.313866 30719 raft_consensus.cc:3060] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.313325 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:48.315768 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:48.315949 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:48.315869 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb7cbe0a9224412880144927ec8fa99" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:48.316075 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:48.316133 28294 raft_consensus.cc:3060] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.316220 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:48.316422 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:48.316603 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:48.316764 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:48.316917 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:48.317348 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
W20260320 15:27:48.317302 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 50516.3us
I20260320 15:27:48.317526 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:48.317688 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:48.317834 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:48.318329 28889 catalog_manager.cc:5671] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } }
I20260320 15:27:48.318684 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:48.318781 28294 raft_consensus.cc:2468] T 2cb7cbe0a9224412880144927ec8fa99 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:48.318914 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:48.319320 28383 leader_election.cc:304] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.319646 30636 raft_consensus.cc:2804] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.319686 30773 raft_consensus.cc:493] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.319878 30636 raft_consensus.cc:697] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:48.319932 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfbc7338eda489987315e681ee44146" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.319839 30773 raft_consensus.cc:515] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.320091 28439 raft_consensus.cc:2468] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.320164 30636 consensus_queue.cc:237] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.320643 30773 leader_election.cc:290] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.321054 28240 leader_election.cc:304] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.321761 30773 raft_consensus.cc:2804] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.321871 30773 raft_consensus.cc:493] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.321767 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197f1935099140a98c5eb801cf44046a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.321946 30773 raft_consensus.cc:3060] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.322044 28439 raft_consensus.cc:2468] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.323091 28413 tablet_service.cc:1511] Processing CreateTablet for tablet d38a43965b1545d29041bedd99f13619 (DEFAULT_TABLE table=default.Table_44d6e0bede5f4c87971a0e127aef1be0 [id=ddfa59d1ea4e4064a54f5e9bbe73d51c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.323518 28413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d38a43965b1545d29041bedd99f13619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.323915 28411 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc8e0fa469446fcac5ed9fb048cd431 (DEFAULT_TABLE table=default.Table_180c8e69093147538db5cd1b8489fa6b [id=cbff53e2a2fb46c2a2881bc0f962262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.324326 28411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc8e0fa469446fcac5ed9fb048cd431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.324626 30773 raft_consensus.cc:515] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.325269 30773 leader_election.cc:290] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.326406 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 068515f077624aed83309109eb59d077 (DEFAULT_TABLE table=default.Table_fe5422b8425b4fe9988570705b9fddcd [id=6e984ba06fc44c889a663e50b762d4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.326797 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068515f077624aed83309109eb59d077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.326759 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3d9cd2c074431194d805c51ba1da44 (DEFAULT_TABLE table=default.Table_228877df377e4f8bbe522f4d57092b2e [id=e95339498f7b4d1ba1e6b83c26ff39f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.327054 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d9cd2c074431194d805c51ba1da44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.327291 28424 tablet_service.cc:1511] Processing CreateTablet for tablet e61e0f9764fe4049975b65f6221feb78 (DEFAULT_TABLE table=default.Table_3aeb90fb3c79479ea771ab8d88f6bbf4 [id=f271402e7ac84029be88aeae6beddfe9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.327632 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e61e0f9764fe4049975b65f6221feb78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.327786 28422 tablet_service.cc:1511] Processing CreateTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 (DEFAULT_TABLE table=default.Table_4478d6addfd34e128e575a94d94c09f3 [id=248f8c828a0e46d7ae560e84fdef293c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.328095 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c37fc78b5d1546f69723cac88ce26ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.328912 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd50f4df4b34c8d8f3460514533508a (DEFAULT_TABLE table=default.Table_74ed94505ea049559e4e388efe568e9b [id=4cadab46e6394618ada8be7ebe446f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.329226 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd50f4df4b34c8d8f3460514533508a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:48.330260 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55bd68c6e4364a038a815d1fc05ad25e consensus state has no leader
I20260320 15:27:48.331916 28426 tablet_service.cc:1511] Processing CreateTablet for tablet c4c8511f0fc9412ca9754ffa9aa84c8b (DEFAULT_TABLE table=default.Table_8af7bea7b0024b3c93158cfcb8d04bbc [id=589823637776495fa71481920826e341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.332298 28426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c8511f0fc9412ca9754ffa9aa84c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.337956 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4190329df1b644c392ff8cbace9aae7a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.338387 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfbc7338eda489987315e681ee44146" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.338552 28550 tablet_service.cc:1511] Processing CreateTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 (DEFAULT_TABLE table=default.Table_4478d6addfd34e128e575a94d94c09f3 [id=248f8c828a0e46d7ae560e84fdef293c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:48.338649 28239 leader_election.cc:343] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.338806 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 7fc8e0fa469446fcac5ed9fb048cd431 (DEFAULT_TABLE table=default.Table_180c8e69093147538db5cd1b8489fa6b [id=cbff53e2a2fb46c2a2881bc0f962262a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.338860 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfbc7338eda489987315e681ee44146" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.338997 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c37fc78b5d1546f69723cac88ce26ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.339144 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7fc8e0fa469446fcac5ed9fb048cd431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.339335 28555 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3d9cd2c074431194d805c51ba1da44 (DEFAULT_TABLE table=default.Table_228877df377e4f8bbe522f4d57092b2e [id=e95339498f7b4d1ba1e6b83c26ff39f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.339665 28555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3d9cd2c074431194d805c51ba1da44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.343060 28560 tablet_service.cc:1511] Processing CreateTablet for tablet c4c8511f0fc9412ca9754ffa9aa84c8b (DEFAULT_TABLE table=default.Table_8af7bea7b0024b3c93158cfcb8d04bbc [id=589823637776495fa71481920826e341]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.343570 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c8511f0fc9412ca9754ffa9aa84c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.338557 28575 raft_consensus.cc:2468] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.350499 28240 leader_election.cc:304] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.351128 29643 ts_tablet_manager.cc:1929] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.351243 29643 log.cc:1199] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:48.351732 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfbc7338eda489987315e681ee44146" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.351938 28439 raft_consensus.cc:3060] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.352612 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:48.353457 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:48.353852 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:48.354022 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:48.354470 28439 raft_consensus.cc:2468] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.354617 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:48.355190 30623 raft_consensus.cc:2804] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.355301 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:48.355316 30623 raft_consensus.cc:493] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.355443 30623 raft_consensus.cc:3060] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.359447 28889 catalog_manager.cc:5671] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } }
I20260320 15:27:48.359659 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb7cbe0a9224412880144927ec8fa99" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.359901 28576 raft_consensus.cc:3060] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.359889 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:48.360131 28734 raft_consensus.cc:740] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.360878 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:48.361156 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:48.362074 29640 ts_tablet_manager.cc:1929] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.362191 29640 log.cc:1199] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/3ccbf4f32636463f8e1e005a629b92cc
I20260320 15:27:48.362591 29640 ts_tablet_manager.cc:1950] T 3ccbf4f32636463f8e1e005a629b92cc P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:48.362942 28576 raft_consensus.cc:2468] T 2cb7cbe0a9224412880144927ec8fa99 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:48.363405 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 204) took 9062 ms. Trace:
I20260320 15:27:48.365576 28240 leader_election.cc:304] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.366658 30466 raft_consensus.cc:2804] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.366920 30466 raft_consensus.cc:697] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.367264 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:48.367270 30466 consensus_queue.cc:237] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.367486 29643 ts_tablet_manager.cc:1950] T cc97a5f2bd964a05bb7b2427226debdd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.368124 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:48.368366 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 368) took 2995 ms. Trace:
I20260320 15:27:48.368451 29643 rpcz_store.cc:276] 0320 15:27:45.373067 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.390564 (+ 17497us) service_pool.cc:225] Handling call
0320 15:27:48.252209 (+2861645us) ts_tablet_manager.cc:1151] Deleting tablet cc97a5f2bd964a05bb7b2427226debdd
0320 15:27:48.252230 (+ 21us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.327870 (+ 75640us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.351020 (+ 23150us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.368345 (+ 17325us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2849119}
I20260320 15:27:48.368587 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
W20260320 15:27:48.368942 28239 leader_election.cc:343] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a
I20260320 15:27:48.372545 28734 consensus_queue.cc:260] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.374568 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet cc97a5f2bd964a05bb7b2427226debdd (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
I20260320 15:27:48.374819 28734 ts_tablet_manager.cc:1434] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.095s user 0.018s sys 0.000s
I20260320 15:27:48.373142 30456 raft_consensus.cc:493] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.375200 28734 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.375155 30456 raft_consensus.cc:515] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.375798 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:48.375851 30456 leader_election.cc:290] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.376080 28889 catalog_manager.cc:5671] T 85c397eab0ed4df188b0dc2a58077b39 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } }
I20260320 15:27:48.376241 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db16d86664794305bde714a7ff4d5caf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.376544 28889 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:48.376682 29643 tablet_replica.cc:333] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
W20260320 15:27:48.376814 28382 leader_election.cc:343] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.376891 29643 raft_consensus.cc:2243] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.377031 29643 raft_consensus.cc:2272] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.378705 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db16d86664794305bde714a7ff4d5caf" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:48.379377 28383 leader_election.cc:343] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.379506 28383 leader_election.cc:304] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.381523 29643 ts_tablet_manager.cc:1916] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.381954 30767 raft_consensus.cc:2749] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.363492 29640 rpcz_store.cc:276] 0320 15:27:39.300957 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.301013 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:48.271694 (+8970681us) ts_tablet_manager.cc:1151] Deleting tablet 3ccbf4f32636463f8e1e005a629b92cc
0320 15:27:48.271704 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.333426 (+ 61722us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.361950 (+ 28524us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.363387 (+ 1437us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8970518}
I20260320 15:27:48.384328 30719 raft_consensus.cc:515] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.385162 30719 leader_election.cc:290] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.384465 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 205) took 9062 ms. Trace:
I20260320 15:27:48.385527 29640 rpcz_store.cc:276] 0320 15:27:39.322035 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.322159 (+ 124us) service_pool.cc:225] Handling call
0320 15:27:48.384350 (+9062191us) ts_tablet_manager.cc:1151] Deleting tablet 5bc3fb47136d4b88a2898af399d74a8a
0320 15:27:48.384363 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.384446 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9061963}
I20260320 15:27:48.386082 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4190329df1b644c392ff8cbace9aae7a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.386576 28734 tablet_bootstrap.cc:654] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.386771 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 3ccbf4f32636463f8e1e005a629b92cc (table default.Table_86f6401b78c540eea6cb427c9bd38f6f [id=c4c05d3551154216921458f581aaf0d1]) successfully deleted
I20260320 15:27:48.387063 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4190329df1b644c392ff8cbace9aae7a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.388731 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 206) took 9065 ms. Trace:
I20260320 15:27:48.389478 29640 rpcz_store.cc:276] 0320 15:27:39.323065 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.323186 (+ 121us) service_pool.cc:225] Handling call
0320 15:27:48.388612 (+9065426us) ts_tablet_manager.cc:1151] Deleting tablet 931f6f88a1d94a40b69f07216b8313c8
0320 15:27:48.388625 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.388711 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9065189}
I20260320 15:27:48.390728 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 207) took 9023 ms. Trace:
I20260320 15:27:48.391363 29640 rpcz_store.cc:276] 0320 15:27:39.379758 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.379820 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:48.390638 (+9010818us) ts_tablet_manager.cc:1151] Deleting tablet a4bfb87d5d5741adb0ad2f2a88f4867f
0320 15:27:48.390649 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.390712 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9010580}
W20260320 15:27:48.391785 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 931f6f88a1d94a40b69f07216b8313c8 because the tablet was not found. No further retry: Not found: Tablet not found: 931f6f88a1d94a40b69f07216b8313c8
W20260320 15:27:48.396193 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet a4bfb87d5d5741adb0ad2f2a88f4867f because the tablet was not found. No further retry: Not found: Tablet not found: a4bfb87d5d5741adb0ad2f2a88f4867f
I20260320 15:27:48.396857 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 208) took 9016 ms. Trace:
I20260320 15:27:48.399132 29640 rpcz_store.cc:276] 0320 15:27:39.380334 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.380406 (+ 72us) service_pool.cc:225] Handling call
0320 15:27:48.396719 (+9016313us) ts_tablet_manager.cc:1151] Deleting tablet f5f92561764b4cb9b15bb9a310a350f0
0320 15:27:48.396736 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.396829 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9016144}
W20260320 15:27:48.400982 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet f5f92561764b4cb9b15bb9a310a350f0 because the tablet was not found. No further retry: Not found: Tablet not found: f5f92561764b4cb9b15bb9a310a350f0
I20260320 15:27:48.401806 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 209) took 9021 ms. Trace:
I20260320 15:27:48.403206 29640 rpcz_store.cc:276] 0320 15:27:39.380897 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.381023 (+ 126us) service_pool.cc:225] Handling call
0320 15:27:48.401684 (+9020661us) ts_tablet_manager.cc:1151] Deleting tablet edb7b53bb7444a39b471218c6039542f
0320 15:27:48.401698 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.401782 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9020504}
W20260320 15:27:48.405179 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet edb7b53bb7444a39b471218c6039542f because the tablet was not found. No further retry: Not found: Tablet not found: edb7b53bb7444a39b471218c6039542f
I20260320 15:27:48.406031 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 210) took 9024 ms. Trace:
I20260320 15:27:48.407435 29640 rpcz_store.cc:276] 0320 15:27:39.381563 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.381622 (+ 59us) service_pool.cc:225] Handling call
0320 15:27:48.405909 (+9024287us) ts_tablet_manager.cc:1151] Deleting tablet 1c2d7e4d26994a17bf2d2615d6552412
0320 15:27:48.405924 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.406008 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9024130}
W20260320 15:27:48.409601 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 1c2d7e4d26994a17bf2d2615d6552412 because the tablet was not found. No further retry: Not found: Tablet not found: 1c2d7e4d26994a17bf2d2615d6552412
I20260320 15:27:48.410804 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 211) took 9028 ms. Trace:
I20260320 15:27:48.412811 29640 rpcz_store.cc:276] 0320 15:27:39.382816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.382923 (+ 107us) service_pool.cc:225] Handling call
0320 15:27:48.410656 (+9027733us) ts_tablet_manager.cc:1151] Deleting tablet 757b5afdea674217909610e3719a8804
0320 15:27:48.410670 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.410775 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9027565}
W20260320 15:27:48.415076 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 757b5afdea674217909610e3719a8804 because the tablet was not found. No further retry: Not found: Tablet not found: 757b5afdea674217909610e3719a8804
W20260320 15:27:48.387442 28240 leader_election.cc:343] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Illegal state: Tablet not RUNNING: STOPPED
W20260320 15:27:48.434264 28240 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 46798.6us
W20260320 15:27:48.434417 28239 leader_election.cc:343] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.430784 30623 raft_consensus.cc:515] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.440127 28239 leader_election.cc:304] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.440555 30623 leader_election.cc:290] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:48.440666 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 52595.1us
I20260320 15:27:48.440761 30623 raft_consensus.cc:2749] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.441745 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "197f1935099140a98c5eb801cf44046a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.441959 28439 raft_consensus.cc:3060] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.442107 28280 tablet_service.cc:1511] Processing CreateTablet for tablet b6d5e4ff66914ef2a1193489bcf8f20a (DEFAULT_TABLE table=default.Table_b74d0dcbf6da41dfbf9b6bad57798e6d [id=7cde1d20c04e43cfa18db3eacb20ac60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.442678 28280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d5e4ff66914ef2a1193489bcf8f20a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:48.444382 28239 leader_election.cc:343] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a
I20260320 15:27:48.444788 28439 raft_consensus.cc:2468] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.446743 28240 leader_election.cc:304] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.447209 30466 raft_consensus.cc:2804] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.447386 30466 raft_consensus.cc:697] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.448086 30466 consensus_queue.cc:237] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.449368 28560 tablet_service.cc:1511] Processing CreateTablet for tablet e44031268eb54d2796040168e7a37e66 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.449815 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44031268eb54d2796040168e7a37e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.450155 28276 tablet_service.cc:1511] Processing CreateTablet for tablet e44031268eb54d2796040168e7a37e66 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.450475 28421 tablet_service.cc:1511] Processing CreateTablet for tablet 31ec989bdd1e4e4eadbe011ada114fbf (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.450608 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44031268eb54d2796040168e7a37e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.450846 28421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ec989bdd1e4e4eadbe011ada114fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.451779 28424 tablet_service.cc:1511] Processing CreateTablet for tablet b6d5e4ff66914ef2a1193489bcf8f20a (DEFAULT_TABLE table=default.Table_b74d0dcbf6da41dfbf9b6bad57798e6d [id=7cde1d20c04e43cfa18db3eacb20ac60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.452095 28424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d5e4ff66914ef2a1193489bcf8f20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.455152 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "547d82cee19b4959b3a779c8b8088e83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.455353 28447 raft_consensus.cc:3060] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.455847 28409 tablet_service.cc:1511] Processing CreateTablet for tablet e44031268eb54d2796040168e7a37e66 (DEFAULT_TABLE table=default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.456245 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44031268eb54d2796040168e7a37e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.457826 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 31ec989bdd1e4e4eadbe011ada114fbf (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.458325 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ec989bdd1e4e4eadbe011ada114fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.460036 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 31ec989bdd1e4e4eadbe011ada114fbf (DEFAULT_TABLE table=default.Table_401dcb9cf68646098dd34f567b99c2f9 [id=b1795f75826943ffacab54cf632261a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.460632 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31ec989bdd1e4e4eadbe011ada114fbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.463848 30783 raft_consensus.cc:493] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.463997 30783 raft_consensus.cc:515] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.464630 30783 leader_election.cc:290] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.465324 28281 tablet_service.cc:1511] Processing CreateTablet for tablet f90f8f20637d4d0cb06cee5560f5dad9 (DEFAULT_TABLE table=default.Table_5ef08b954dbd41f7b7ff75d7d07515dd [id=465a7290c2114ad1951cf5df366205ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.465852 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90f8f20637d4d0cb06cee5560f5dad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.468080 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.471870 30783 raft_consensus.cc:493] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.472368 30783 raft_consensus.cc:515] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.473161 30783 leader_election.cc:290] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.474531 28447 raft_consensus.cc:2468] T 547d82cee19b4959b3a779c8b8088e83 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.475323 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 212) took 9091 ms. Trace:
I20260320 15:27:48.476059 29640 rpcz_store.cc:276] 0320 15:27:39.383656 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.383743 (+ 87us) service_pool.cc:225] Handling call
0320 15:27:48.475169 (+9091426us) ts_tablet_manager.cc:1151] Deleting tablet 993e2fb1561f4317be85e9b5e5816edc
0320 15:27:48.475186 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.475298 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9091248}
I20260320 15:27:48.476541 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.477658 28439 raft_consensus.cc:2468] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:48.479197 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2243e6811744b7cba99a487f8e8bb83" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.479491 28302 raft_consensus.cc:2393] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
W20260320 15:27:48.477833 28524 leader_election.cc:343] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:48.484623 28524 leader_election.cc:304] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:48.485082 28524 leader_election.cc:304] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:48.485529 28524 leader_election.cc:343] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.486019 28524 leader_election.cc:304] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.486240 30725 raft_consensus.cc:2804] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.486367 30725 raft_consensus.cc:697] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.486706 30630 raft_consensus.cc:3060] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.486641 30725 consensus_queue.cc:237] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.486375 30655 raft_consensus.cc:2804] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.487797 30655 raft_consensus.cc:493] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.487885 30655 raft_consensus.cc:3060] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.489008 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:48.489336 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:48.489585 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:48.489712 30630 raft_consensus.cc:2749] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.489755 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:48.490017 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:48.490550 30655 raft_consensus.cc:515] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.490722 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:48.491009 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:48.491175 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:48.491196 30655 leader_election.cc:290] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.491328 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
W20260320 15:27:48.492151 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 993e2fb1561f4317be85e9b5e5816edc because the tablet was not found. No further retry: Not found: Tablet not found: 993e2fb1561f4317be85e9b5e5816edc
I20260320 15:27:48.492339 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 213) took 9108 ms. Trace:
I20260320 15:27:48.493206 30655 consensus_queue.cc:1048] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.495142 29640 rpcz_store.cc:276] 0320 15:27:39.384515 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.384615 (+ 100us) service_pool.cc:225] Handling call
0320 15:27:48.490682 (+9106067us) ts_tablet_manager.cc:1151] Deleting tablet 3e7b964395f24e45bc2715be3fac6dbe
0320 15:27:48.490694 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.492317 (+ 1623us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9105910}
I20260320 15:27:48.496996 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:48.497831 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "514e7b02a28142c2b371644089044a9d" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.498450 28447 raft_consensus.cc:3060] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.497689 28524 leader_election.cc:343] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:48.499049 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 3e7b964395f24e45bc2715be3fac6dbe because the tablet was not found. No further retry: Not found: Tablet not found: 3e7b964395f24e45bc2715be3fac6dbe
I20260320 15:27:48.499553 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 214) took 9114 ms. Trace:
I20260320 15:27:48.500442 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:48.500823 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:48.501030 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:48.501423 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:48.501863 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:48.502331 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:48.502662 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:48.502813 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:48.503913 29640 rpcz_store.cc:276] 0320 15:27:39.414826 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.414952 (+ 126us) service_pool.cc:225] Handling call
0320 15:27:48.499422 (+9084470us) ts_tablet_manager.cc:1151] Deleting tablet 66a9ddea49e641da881fba339ddef549
0320 15:27:48.499434 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.499506 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9084196}
I20260320 15:27:48.505062 28422 tablet_service.cc:1511] Processing CreateTablet for tablet f90f8f20637d4d0cb06cee5560f5dad9 (DEFAULT_TABLE table=default.Table_5ef08b954dbd41f7b7ff75d7d07515dd [id=465a7290c2114ad1951cf5df366205ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.505556 28422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90f8f20637d4d0cb06cee5560f5dad9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:48.504673 28524 consensus_peers.cc:597] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:48.507715 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 66a9ddea49e641da881fba339ddef549 because the tablet was not found. No further retry: Not found: Tablet not found: 66a9ddea49e641da881fba339ddef549
I20260320 15:27:48.508355 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 215) took 9087 ms. Trace:
I20260320 15:27:48.509303 29643 ts_tablet_manager.cc:1929] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:48.512501 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:48.512861 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:48.514038 30623 raft_consensus.cc:493] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.514293 30623 raft_consensus.cc:515] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.514688 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:48.515071 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:48.515214 30623 leader_election.cc:290] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.511938 28562 tablet_service.cc:1511] Processing CreateTablet for tablet b6d5e4ff66914ef2a1193489bcf8f20a (DEFAULT_TABLE table=default.Table_b74d0dcbf6da41dfbf9b6bad57798e6d [id=7cde1d20c04e43cfa18db3eacb20ac60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.516436 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:48.517236 28239 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.516563 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6d5e4ff66914ef2a1193489bcf8f20a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.512588 29640 rpcz_store.cc:276] 0320 15:27:39.420546 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.423685 (+ 3139us) service_pool.cc:225] Handling call
0320 15:27:48.508232 (+9084547us) ts_tablet_manager.cc:1151] Deleting tablet c752fb99ef494bda9d144d0e180e3100
0320 15:27:48.508246 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.508331 (+ 85us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9084288}
W20260320 15:27:48.518851 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet c752fb99ef494bda9d144d0e180e3100 because the tablet was not found. No further retry: Not found: Tablet not found: c752fb99ef494bda9d144d0e180e3100
I20260320 15:27:48.519312 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 216) took 8696 ms. Trace:
I20260320 15:27:48.498457 28556 tablet_service.cc:1511] Processing CreateTablet for tablet f90f8f20637d4d0cb06cee5560f5dad9 (DEFAULT_TABLE table=default.Table_5ef08b954dbd41f7b7ff75d7d07515dd [id=465a7290c2114ad1951cf5df366205ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:48.509443 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:48.519806 29643 log.cc:1199] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/4190329df1b644c392ff8cbace9aae7a
I20260320 15:27:48.519951 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f90f8f20637d4d0cb06cee5560f5dad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:48.520460 29643 ts_tablet_manager.cc:1950] T 4190329df1b644c392ff8cbace9aae7a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.520962 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 369) took 2777 ms. Trace:
I20260320 15:27:48.521225 29643 rpcz_store.cc:276] 0320 15:27:45.743487 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.743657 (+ 170us) service_pool.cc:225] Handling call
0320 15:27:48.376627 (+2632970us) ts_tablet_manager.cc:1151] Deleting tablet 4190329df1b644c392ff8cbace9aae7a
0320 15:27:48.376638 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.393722 (+ 17084us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.509234 (+115512us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.520945 (+ 11711us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2632740}
I20260320 15:27:48.521828 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]) successfully deleted
I20260320 15:27:48.522022 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:48.522337 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:48.522395 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 370) took 2751 ms. Trace:
I20260320 15:27:48.522586 29643 rpcz_store.cc:276] 0320 15:27:45.770508 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.771158 (+ 650us) service_pool.cc:225] Handling call
0320 15:27:48.522249 (+2751091us) ts_tablet_manager.cc:1151] Deleting tablet d04ab44039124f36ac60601098586b4e
0320 15:27:48.522261 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.522375 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2750832}
I20260320 15:27:48.522964 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]). Expected version 1 got 0
W20260320 15:27:48.523173 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d04ab44039124f36ac60601098586b4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d04ab44039124f36ac60601098586b4e already in progress: creating tablet
I20260320 15:27:48.523447 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:48.523751 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 377) took 2342 ms. Trace:
I20260320 15:27:48.523875 29643 rpcz_store.cc:276] 0320 15:27:46.181393 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.181575 (+ 182us) service_pool.cc:225] Handling call
0320 15:27:48.523506 (+2341931us) ts_tablet_manager.cc:1151] Deleting tablet d823ab41194542fba5540e53e9f4680b
0320 15:27:48.523620 (+ 114us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.523732 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2341755}
I20260320 15:27:48.524644 29643 tablet_replica.cc:333] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.524881 29643 raft_consensus.cc:2243] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.525189 29643 raft_consensus.cc:2272] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:48.524434 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d823ab41194542fba5540e53e9f4680b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d823ab41194542fba5540e53e9f4680b already in progress: creating tablet
I20260320 15:27:48.525640 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:48.526041 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:48.526396 29643 ts_tablet_manager.cc:1916] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.526860 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:48.527161 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:48.527420 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:48.527686 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:48.528116 28901 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:48.533663 28736 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.533870 28736 ts_tablet_manager.cc:1403] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.275s user 0.017s sys 0.033s
I20260320 15:27:48.534643 28736 raft_consensus.cc:359] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.534860 28736 raft_consensus.cc:385] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.534926 28736 raft_consensus.cc:740] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.516963 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.535204 28736 consensus_queue.cc:260] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.535256 28439 raft_consensus.cc:2468] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.535825 28736 ts_tablet_manager.cc:1434] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:48.535928 28240 leader_election.cc:304] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.536255 28736 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.536700 30623 raft_consensus.cc:2804] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.536831 30623 raft_consensus.cc:493] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.536932 30623 raft_consensus.cc:3060] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.538702 28736 tablet_bootstrap.cc:654] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.541340 28734 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.541509 28734 ts_tablet_manager.cc:1403] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.166s user 0.006s sys 0.004s
I20260320 15:27:48.542225 28734 raft_consensus.cc:359] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.542418 28734 raft_consensus.cc:385] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.542471 28734 raft_consensus.cc:740] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.542721 28734 consensus_queue.cc:260] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.543331 28734 ts_tablet_manager.cc:1434] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:48.543646 28734 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.544740 30623 raft_consensus.cc:515] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.545068 30456 raft_consensus.cc:493] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.545225 30456 raft_consensus.cc:515] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.545909 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.512228 28447 raft_consensus.cc:2468] T 514e7b02a28142c2b371644089044a9d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.546046 30456 leader_election.cc:290] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.546434 28524 leader_election.cc:304] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:48.547006 30623 leader_election.cc:290] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.548537 30725 raft_consensus.cc:2804] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.548662 30725 raft_consensus.cc:697] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.548892 30725 consensus_queue.cc:237] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.552183 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae3422cf33324ab89a2b44e27cae0eb2" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.552241 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:48.552865 28239 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:48.552877 28382 leader_election.cc:343] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.553979 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.554144 28733 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.554287 28733 ts_tablet_manager.cc:1403] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.353s user 0.009s sys 0.013s
I20260320 15:27:48.554965 28733 raft_consensus.cc:359] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.555177 28733 raft_consensus.cc:385] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.555240 28733 raft_consensus.cc:740] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.519402 29640 rpcz_store.cc:276] 0320 15:27:39.822918 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:39.822992 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:48.519162 (+8696170us) ts_tablet_manager.cc:1151] Deleting tablet 48c1002cbdd845e2b96dcb57c450cc6d
0320 15:27:48.519174 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.519287 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8695971}
I20260320 15:27:48.555850 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 218) took 8235 ms. Trace:
W20260320 15:27:48.555832 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 48c1002cbdd845e2b96dcb57c450cc6d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48c1002cbdd845e2b96dcb57c450cc6d already in progress: creating tablet
I20260320 15:27:48.556845 29640 rpcz_store.cc:276] 0320 15:27:40.320309 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.320380 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:48.555745 (+8235365us) ts_tablet_manager.cc:1151] Deleting tablet bcec6175c1ca49f5b682a29d5c1e235a
0320 15:27:48.555755 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.555832 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8235068}
I20260320 15:27:48.546063 28439 raft_consensus.cc:3060] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.557613 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet bcec6175c1ca49f5b682a29d5c1e235a because the tablet was not found. No further retry: Not found: Tablet not found: bcec6175c1ca49f5b682a29d5c1e235a
I20260320 15:27:48.558151 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 217) took 8238 ms. Trace:
I20260320 15:27:48.561000 28439 raft_consensus.cc:2468] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.564471 28240 leader_election.cc:304] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.565167 30623 raft_consensus.cc:2804] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.565320 30623 raft_consensus.cc:697] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.565662 30623 consensus_queue.cc:237] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.572474 28383 leader_election.cc:343] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.572701 28383 leader_election.cc:304] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.572618 29640 rpcz_store.cc:276] 0320 15:27:40.320172 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.320651 (+ 479us) service_pool.cc:225] Handling call
0320 15:27:48.558039 (+8237388us) ts_tablet_manager.cc:1151] Deleting tablet f70530cd63a0420fb51527759c951e31
0320 15:27:48.558053 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.558129 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8237180}
I20260320 15:27:48.575567 30456 raft_consensus.cc:2749] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:48.577180 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet f70530cd63a0420fb51527759c951e31 because the tablet was not found. No further retry: Not found: Tablet not found: f70530cd63a0420fb51527759c951e31
I20260320 15:27:48.578629 28736 tablet_bootstrap.cc:492] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.578820 28736 ts_tablet_manager.cc:1403] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260320 15:27:48.579301 30767 raft_consensus.cc:493] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.579439 30767 raft_consensus.cc:515] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.579594 30773 raft_consensus.cc:493] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.579757 30773 raft_consensus.cc:515] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.580087 30767 leader_election.cc:290] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.580456 30773 leader_election.cc:290] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.580864 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.581130 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c1002cbdd845e2b96dcb57c450cc6d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.581334 28439 raft_consensus.cc:2468] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.581465 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c1002cbdd845e2b96dcb57c450cc6d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.581670 28736 raft_consensus.cc:359] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.581732 30773 consensus_queue.cc:1048] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.582000 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 219) took 8261 ms. Trace:
I20260320 15:27:48.582064 28736 raft_consensus.cc:385] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.582172 28736 raft_consensus.cc:740] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.582352 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594357400f8c44138c7c4f11bf6d5afd" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.582196 29640 rpcz_store.cc:276] 0320 15:27:40.320448 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.320719 (+ 271us) service_pool.cc:225] Handling call
0320 15:27:48.581913 (+8261194us) ts_tablet_manager.cc:1151] Deleting tablet c323b7bcb671464ea6ff584a633f10c0
0320 15:27:48.581923 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.581982 (+ 59us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8261013}
I20260320 15:27:48.582401 28736 consensus_queue.cc:260] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.582620 28297 raft_consensus.cc:2393] T 594357400f8c44138c7c4f11bf6d5afd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22e73471b9046d28d3cf63c905b11b7 in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
W20260320 15:27:48.582823 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet c323b7bcb671464ea6ff584a633f10c0 because the tablet was not found. No further retry: Not found: Tablet not found: c323b7bcb671464ea6ff584a633f10c0
I20260320 15:27:48.582945 28736 ts_tablet_manager.cc:1434] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:48.583240 28736 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.583330 28240 leader_election.cc:304] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.583724 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 220) took 8263 ms. Trace:
I20260320 15:27:48.583859 29640 rpcz_store.cc:276] 0320 15:27:40.320596 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.320977 (+ 381us) service_pool.cc:225] Handling call
0320 15:27:48.583632 (+8262655us) ts_tablet_manager.cc:1151] Deleting tablet bb7a910bd2904a40ae63608377206c7e
0320 15:27:48.583644 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.583706 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8262482}
I20260320 15:27:48.583989 30773 raft_consensus.cc:2804] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.584076 30773 raft_consensus.cc:493] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.584189 30773 raft_consensus.cc:3060] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.584539 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet bb7a910bd2904a40ae63608377206c7e because the tablet was not found. No further retry: Not found: Tablet not found: bb7a910bd2904a40ae63608377206c7e
I20260320 15:27:48.585100 29640 tablet_replica.cc:333] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:48.585374 29640 raft_consensus.cc:2243] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.585633 29640 raft_consensus.cc:2272] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.585908 28736 tablet_bootstrap.cc:654] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.586474 29640 ts_tablet_manager.cc:1916] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.587311 30773 raft_consensus.cc:515] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.576254 28733 consensus_queue.cc:260] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.588070 28382 leader_election.cc:304] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:48.588022 28239 leader_election.cc:343] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.588233 28733 ts_tablet_manager.cc:1434] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.034s user 0.001s sys 0.001s
I20260320 15:27:48.588637 30767 raft_consensus.cc:3060] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.589464 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c1002cbdd845e2b96dcb57c450cc6d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.589635 28447 raft_consensus.cc:3060] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.591358 30767 raft_consensus.cc:2749] T 594357400f8c44138c7c4f11bf6d5afd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.592203 28447 raft_consensus.cc:2468] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.593334 30773 leader_election.cc:290] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.593662 28240 leader_election.cc:304] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.593858 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48c1002cbdd845e2b96dcb57c450cc6d" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.594125 30466 raft_consensus.cc:2804] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.594312 30466 raft_consensus.cc:697] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.594625 30466 consensus_queue.cc:237] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.596020 28239 leader_election.cc:343] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.596233 28733 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:48.597601 28904 catalog_manager.cc:5671] T ea7f260f4edd4fc082ab82880143fa31 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.598129 28733 tablet_bootstrap.cc:654] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.599094 28904 catalog_manager.cc:5671] T c3a49e310b524f3b87eb89f3e84b7800 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.601644 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:48.602104 28904 catalog_manager.cc:5671] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.603771 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:48.561053 29643 ts_tablet_manager.cc:1929] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.606477 29643 log.cc:1199] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:48.607416 29643 ts_tablet_manager.cc:1950] T 90cd18e2ccbf48c8b804329bd3abf847 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.609413 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 378) took 2427 ms. Trace:
I20260320 15:27:48.610997 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:48.612233 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:48.612656 28904 catalog_manager.cc:5671] T 11bef661385b4221a991710ffb34dccb P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.614704 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:48.615604 29643 rpcz_store.cc:276] 0320 15:27:46.181529 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.181802 (+ 273us) service_pool.cc:225] Handling call
0320 15:27:48.524597 (+2342795us) ts_tablet_manager.cc:1151] Deleting tablet 90cd18e2ccbf48c8b804329bd3abf847
0320 15:27:48.524608 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.557722 (+ 33114us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.560931 (+ 3209us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.609389 (+ 48458us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2342676}
I20260320 15:27:48.616860 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 90cd18e2ccbf48c8b804329bd3abf847 (table default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]) successfully deleted
I20260320 15:27:48.616978 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 379) took 2415 ms. Trace:
I20260320 15:27:48.617157 30395 raft_consensus.cc:493] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.617090 29643 rpcz_store.cc:276] 0320 15:27:46.201658 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.201725 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:48.616810 (+2415085us) ts_tablet_manager.cc:1151] Deleting tablet b16ffb28029e4b7ca37ce36efe9e0271
0320 15:27:48.616822 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.616958 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2414893}
I20260320 15:27:48.617448 30395 raft_consensus.cc:515] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.617563 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b16ffb28029e4b7ca37ce36efe9e0271 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16ffb28029e4b7ca37ce36efe9e0271 already in progress: creating tablet
I20260320 15:27:48.618475 30395 leader_election.cc:290] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.619213 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20efd94b09746c88bb48ba36af4759e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.619227 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20efd94b09746c88bb48ba36af4759e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.619328 28734 tablet_bootstrap.cc:654] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.619418 28447 raft_consensus.cc:2468] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:48.619974 28524 leader_election.cc:304] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:48.620651 28524 leader_election.cc:343] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.617228 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:48.621183 30395 raft_consensus.cc:2804] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.621340 30395 raft_consensus.cc:493] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.621400 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:48.621481 30395 raft_consensus.cc:3060] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.621731 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:48.624912 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:48.625275 29643 tablet_replica.cc:333] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.625870 29643 raft_consensus.cc:2243] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.627285 29643 raft_consensus.cc:2272] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.628551 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:48.630347 29643 ts_tablet_manager.cc:1916] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:48.630992 28239 consensus_peers.cc:597] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:48.630714 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:48.632445 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:48.632794 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:48.635699 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:48.636252 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:48.636865 28736 tablet_bootstrap.cc:492] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.639030 28736 ts_tablet_manager.cc:1403] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.056s user 0.006s sys 0.006s
I20260320 15:27:48.640106 28736 raft_consensus.cc:359] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.640455 28736 raft_consensus.cc:385] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.640604 28736 raft_consensus.cc:740] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.640946 28736 consensus_queue.cc:260] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.641728 28736 ts_tablet_manager.cc:1434] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:48.642668 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:48.643370 28904 catalog_manager.cc:5671] T 31379c9e2adc4fd48e0d0b6e9443615a P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.644047 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:48.644145 28736 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.644496 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:48.644987 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:48.645272 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:48.647773 28736 tablet_bootstrap.cc:654] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.651392 28904 catalog_manager.cc:5671] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.652635 28733 tablet_bootstrap.cc:492] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.652782 28733 ts_tablet_manager.cc:1403] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.057s user 0.002s sys 0.002s
I20260320 15:27:48.653223 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:48.653435 28733 raft_consensus.cc:359] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.653615 28733 raft_consensus.cc:385] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.653667 28733 raft_consensus.cc:740] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.653855 28733 consensus_queue.cc:260] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.654829 29643 ts_tablet_manager.cc:1929] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:48.654944 29643 log.cc:1199] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b4cad900411e4c9686701aa99206dd23
I20260320 15:27:48.655310 29643 ts_tablet_manager.cc:1950] T b4cad900411e4c9686701aa99206dd23 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.655611 30395 raft_consensus.cc:515] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.656003 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:48.656203 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 380) took 2449 ms. Trace:
I20260320 15:27:48.656308 29643 rpcz_store.cc:276] 0320 15:27:46.206491 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.206560 (+ 69us) service_pool.cc:225] Handling call
0320 15:27:48.625215 (+2418655us) ts_tablet_manager.cc:1151] Deleting tablet b4cad900411e4c9686701aa99206dd23
0320 15:27:48.625227 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.646190 (+ 20963us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.654715 (+ 8525us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.656176 (+ 1461us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2418392}
I20260320 15:27:48.656663 29643 tablet_replica.cc:333] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.656867 29643 raft_consensus.cc:2243] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.657176 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b4cad900411e4c9686701aa99206dd23 (table default.Table_e9d83c80d8c64b19af48bda73dff8d8c [id=7b608418906d4f4596846ba336c3c4fd]) successfully deleted
I20260320 15:27:48.657337 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:48.658207 30395 leader_election.cc:290] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.658432 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20efd94b09746c88bb48ba36af4759e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:48.658726 28297 raft_consensus.cc:3060] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.661545 28297 raft_consensus.cc:2468] T b20efd94b09746c88bb48ba36af4759e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.658285 28904 catalog_manager.cc:5671] T 321fe367ec3348f8a049167f3f8c15d5 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.662035 28524 leader_election.cc:304] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:48.662361 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:48.662879 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:48.663350 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:48.664011 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:48.664587 28904 catalog_manager.cc:5671] T 78288f7aea204870af75060476760ee5 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.657503 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20efd94b09746c88bb48ba36af4759e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.673089 28447 raft_consensus.cc:3060] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.673118 29643 raft_consensus.cc:2272] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.673950 29643 ts_tablet_manager.cc:1916] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.674206 28904 catalog_manager.cc:5671] T 8f408aa18d184e5c9a1f900827e70ca8 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.676137 28447 raft_consensus.cc:2468] T b20efd94b09746c88bb48ba36af4759e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.677021 30629 raft_consensus.cc:2804] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.676849 28904 catalog_manager.cc:5671] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:48.677202 30629 raft_consensus.cc:697] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.677554 30629 consensus_queue.cc:237] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.681059 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:48.681488 30630 consensus_queue.cc:1048] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.682237 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:48.685832 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_f81d07130d8943e5bb8ba81230e2e6b7 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 3 got 0
I20260320 15:27:48.686285 28733 ts_tablet_manager.cc:1434] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.033s user 0.001s sys 0.001s
I20260320 15:27:48.686762 28904 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:48.686882 28733 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:48.687796 28903 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 165) took 2127 ms. Trace:
I20260320 15:27:48.687952 28903 rpcz_store.cc:276] 0320 15:27:46.560465 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.130252 (+1569787us) service_pool.cc:225] Handling call
0320 15:27:48.131042 (+ 790us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:48.131306 (+ 264us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:48.131554 (+ 248us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:48.131621 (+ 67us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:48.176636 (+ 45015us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:48.687743 (+511107us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.132866 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.133003 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.133012 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:48.133015 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.133131 (+ 116us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.133148 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.133152 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:48.133220 (+ 68us) tablet.cc:700] Row locks acquired
0320 15:27:48.133222 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:48.133260 (+ 38us) write_op.cc:270] Start()
0320 15:27:48.133311 (+ 51us) write_op.cc:276] Timestamp: P: 1774020468133217 usec, L: 0
0320 15:27:48.133314 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.133576 (+ 262us) log.cc:844] Serialized 405 byte log entry
0320 15:27:48.175071 (+ 41495us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.175210 (+ 139us) write_op.cc:301] APPLY: starting
0320 15:27:48.175232 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.175417 (+ 185us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.175567 (+ 150us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.175646 (+ 79us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.175690 (+ 44us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.175697 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:48.175803 (+ 106us) log.cc:844] Serialized 23 byte log entry
0320 15:27:48.175893 (+ 90us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.175920 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:48.175927 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.176012 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.176017 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":788,"apply.run_wall_time_us":787,"num_ops":1,"prepare.queue_time_us":678,"prepare.run_cpu_time_us":836,"prepare.run_wall_time_us":836,"raft.queue_time_us":223,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":269,"replication_time_us":41692}]]}
I20260320 15:27:48.688772 28734 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.688944 28734 ts_tablet_manager.cc:1403] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.145s user 0.003s sys 0.000s
I20260320 15:27:48.690088 28734 raft_consensus.cc:359] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.690327 28734 raft_consensus.cc:385] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.690385 28734 raft_consensus.cc:740] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.690667 28734 consensus_queue.cc:260] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.700944 28734 ts_tablet_manager.cc:1434] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260320 15:27:48.701468 28734 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.714586 30630 raft_consensus.cc:493] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:48.714787 30630 raft_consensus.cc:515] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.715480 30630 leader_election.cc:290] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:48.717718 28524 leader_election.cc:343] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 35347116811e4aee8142a503dd21843a
W20260320 15:27:48.717958 28524 leader_election.cc:343] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 35347116811e4aee8142a503dd21843a
I20260320 15:27:48.718051 28524 leader_election.cc:304] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.718616 30630 raft_consensus.cc:2749] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:48.721208 28733 tablet_bootstrap.cc:654] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.727406 30630 consensus_queue.cc:1048] T b1b6529e062440828fca2e18b129f8e3 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.731155 30630 raft_consensus.cc:493] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.731463 30630 raft_consensus.cc:515] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.732102 29643 ts_tablet_manager.cc:1929] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:48.732245 29643 log.cc:1199] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/db16d86664794305bde714a7ff4d5caf
I20260320 15:27:48.732489 30630 leader_election.cc:290] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.732641 29643 ts_tablet_manager.cc:1950] T db16d86664794305bde714a7ff4d5caf P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.733487 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 384) took 2352 ms. Trace:
I20260320 15:27:48.733500 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.733924 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4150c4c26a26405b9be16f66d15a8948" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.734079 28437 raft_consensus.cc:2393] T 4150c4c26a26405b9be16f66d15a8948 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:48.735019 28524 leader_election.cc:304] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.735765 30630 raft_consensus.cc:3060] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.737406 28736 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.737608 28736 ts_tablet_manager.cc:1403] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.094s user 0.007s sys 0.009s
I20260320 15:27:48.740458 30630 raft_consensus.cc:2749] T 4150c4c26a26405b9be16f66d15a8948 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:48.740787 28524 consensus_peers.cc:597] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28c453eaf49c4171b0f93d3b26690d83. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:48.744355 28734 tablet_bootstrap.cc:654] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.746172 28889 catalog_manager.cc:6216] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca] alter complete (version 1)
I20260320 15:27:48.752636 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet ad4a30476e7146afae1e77dad078136e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.755993 30466 raft_consensus.cc:493] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.756227 30466 raft_consensus.cc:515] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.757323 30466 leader_election.cc:290] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.757822 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:48.758487 28239 leader_election.cc:343] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.758702 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.758944 28447 raft_consensus.cc:2468] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.759865 28736 raft_consensus.cc:359] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.759974 28240 leader_election.cc:304] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.760085 28736 raft_consensus.cc:385] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.760147 28736 raft_consensus.cc:740] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.760560 30466 raft_consensus.cc:2804] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.760660 30466 raft_consensus.cc:493] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.760737 30466 raft_consensus.cc:3060] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.761873 28902 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 91f393d2ca304b899f788add43da7e9a consensus state has no leader
W20260320 15:27:48.763778 28902 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 102) took 3348 ms (client timeout 1000 ms). Trace:
W20260320 15:27:48.763926 28902 rpcz_store.cc:269] 0320 15:27:45.415107 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.723233 (+308126us) service_pool.cc:225] Handling call
0320 15:27:48.763701 (+3040468us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:46.218143 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:46.218320 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:46.218328 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:46.218332 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:46.218682 (+ 350us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:46.218703 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:46.218711 (+ 8us) tablet.cc:684] Acquiring locks for 5 operations
0320 15:27:46.218945 (+ 234us) tablet.cc:700] Row locks acquired
0320 15:27:46.218950 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:46.219005 (+ 55us) write_op.cc:270] Start()
0320 15:27:46.219052 (+ 47us) write_op.cc:276] Timestamp: P: 1774020466218939 usec, L: 0
0320 15:27:46.219056 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:46.219343 (+ 287us) log.cc:844] Serialized 2024 byte log entry
0320 15:27:46.226240 (+ 6897us) op_driver.cc:464] REPLICATION: finished
0320 15:27:46.226381 (+ 141us) write_op.cc:301] APPLY: starting
0320 15:27:46.226401 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:46.226491 (+ 90us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:46.226531 (+ 40us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:46.226670 (+ 139us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:46.226719 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0320 15:27:46.226726 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:46.226829 (+ 103us) log.cc:844] Serialized 47 byte log entry
0320 15:27:46.226906 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.226942 (+ 36us) write_op.cc:454] Released schema lock
0320 15:27:46.226953 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:46.227045 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:46.227049 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":136459,"reported_tablets":185,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":649,"apply.run_wall_time_us":649,"num_ops":5,"prepare.queue_time_us":562,"prepare.run_cpu_time_us":1350,"prepare.run_wall_time_us":1351,"raft.queue_time_us":578,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":313,"replication_time_us":7136}]]}
I20260320 15:27:48.764895 30466 raft_consensus.cc:515] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.733603 29643 rpcz_store.cc:276] 0320 15:27:46.380574 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.380640 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:48.656608 (+2275968us) ts_tablet_manager.cc:1151] Deleting tablet db16d86664794305bde714a7ff4d5caf
0320 15:27:48.656620 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.698253 (+ 41633us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.727430 (+ 29177us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.733468 (+ 6038us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2275740}
I20260320 15:27:48.765645 29643 tablet_replica.cc:333] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.765928 29643 raft_consensus.cc:2243] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:48.766096 29643 raft_consensus.cc:2272] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.766065 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.766175 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.766291 28447 raft_consensus.cc:3060] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.766489 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.766781 28551 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.766918 28560 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.767036 28559 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.767218 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.767463 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.767935 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.768301 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.768543 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet f0a31d385b81456fb46ea52083874fcf with delete_type TABLET_DATA_DELETED (replaced by tablet 60ad9b381aaa47f1afb63df266faee09) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.768853 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet d547c032332d4d109222b4743fa28561 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.769259 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.769336 28447 raft_consensus.cc:2468] T eb011d672f244585b90272443897ed5a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.769680 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet aafd1e123cd043e7828140d2443e047f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.770148 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.770188 28240 leader_election.cc:304] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.770574 30623 raft_consensus.cc:2804] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.770546 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c2339ff2c474a288876f32b85cecc72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:20 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.770707 30623 raft_consensus.cc:697] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.770983 30623 consensus_queue.cc:237] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.771850 28550 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.772096 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 906f207dee674a4288c8c60b5d3ff9f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.772454 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 442ca186c9d4449dade39271dec5c608 with delete_type TABLET_DATA_DELETED (replaced by tablet 41da3cb06b1147fbae2695bc5c9ffaa5) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:48.772454 27944 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-taskc3SESu/build/asan/bin/kudu pid 29423: Timed out: CheckLeaks RPC to 127.27.74.61:44739 timed out after 1.000s (SENT)
I20260320 15:27:48.772606 27944 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskc3SESu/build/asan/bin/kudu with pid 29423
I20260320 15:27:48.772821 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f0242a7968541f480367b45cda5d9cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:23 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.773253 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 07ff84dedc074dabb4e8e2d8460066e7 with delete_type TABLET_DATA_DELETED (replaced by tablet d94b4a6d75804c3e947af4f588ef8f5a) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.773644 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.773597 28736 consensus_queue.cc:260] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.774000 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.774423 28736 ts_tablet_manager.cc:1434] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20260320 15:27:48.774526 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.774813 28736 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.769337 29643 ts_tablet_manager.cc:1916] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.777154 30466 leader_election.cc:290] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.777623 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:48.778577 28239 leader_election.cc:343] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.769675 29640 ts_tablet_manager.cc:1929] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260320 15:27:48.779994 28734 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:48.780205 28734 ts_tablet_manager.cc:1403] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.009s
I20260320 15:27:48.780391 29640 log.cc:1199] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:48.784260 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]) successfully deleted
I20260320 15:27:48.781003 28734 raft_consensus.cc:359] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.787770 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet d04ab44039124f36ac60601098586b4e with delete_type TABLET_DATA_DELETED (replaced by tablet f77a32615c954da581e9a9e4cb45c497) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.788434 28734 raft_consensus.cc:385] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.788584 28734 raft_consensus.cc:740] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.788919 28734 consensus_queue.cc:260] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:48.789085 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.789701 28734 ts_tablet_manager.cc:1434] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260320 15:27:48.789816 28736 tablet_bootstrap.cc:654] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.790112 28734 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:48.793160 28734 tablet_bootstrap.cc:654] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.793442 30395 raft_consensus.cc:493] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:48.793593 30395 raft_consensus.cc:515] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.794236 30395 leader_election.cc:290] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.795210 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.795857 28524 leader_election.cc:343] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:48.796344 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 8428019c3b074b17aec493c14cc2b911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.797238 28524 leader_election.cc:343] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:48.797411 28524 leader_election.cc:304] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.798051 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.798408 30629 raft_consensus.cc:2749] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:48.798619 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 with delete_type TABLET_DATA_DELETED (replaced by tablet 91f393d2ca304b899f788add43da7e9a) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.800478 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet ae3422cf33324ab89a2b44e27cae0eb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.800629 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet e779a274913f4caeaf82e3f180457830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.800753 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.800943 28280 tablet_service.cc:1558] Processing DeleteTablet for tablet d04ab44039124f36ac60601098586b4e with delete_type TABLET_DATA_DELETED (replaced by tablet f77a32615c954da581e9a9e4cb45c497) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801012 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet a185f84c24ea402684aa1cc3628d4f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801218 28271 tablet_service.cc:1558] Processing DeleteTablet for tablet aba79ddde6e842dfb67597c5d9e371cb with delete_type TABLET_DATA_DELETED (replaced by tablet 71db28879e554f578b8a7b0e14419ab7) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801386 28277 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 with delete_type TABLET_DATA_DELETED (replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801569 28276 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:41 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801734 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.801859 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet e2243e6811744b7cba99a487f8e8bb83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.806085 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 4190329df1b644c392ff8cbace9aae7a with delete_type TABLET_DATA_DELETED (replaced by tablet 806d635bb5594f0bb826b9b31468b485) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.809612 29640 ts_tablet_manager.cc:1950] T 9dbd3b0e92a744dcbc132037c67303ab P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:48.815488 30842 tablet.cc:1723] T 408f5b4d67514838a17c180ae2cd71df P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:48.818395 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 49654c67435a48448794879464f6cfb2 with delete_type TABLET_DATA_DELETED (replaced by tablet e44031268eb54d2796040168e7a37e66) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.819037 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 4cfad5cf843a437e8e15c7236d1dc5fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.819269 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 7840b529206d482283cc550d2c5e7606 with delete_type TABLET_DATA_DELETED (replaced by tablet 31ec989bdd1e4e4eadbe011ada114fbf) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.819667 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.820102 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 889d620c311a42039c22af53ad0802f2 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f94140a5e8b4a06890bca86456d64a6) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.820483 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.820775 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ccbf4f32636463f8e1e005a629b92cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.821084 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet d823ab41194542fba5540e53e9f4680b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.821688 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.822490 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 87cb35cefee841d0881b08c456df5a97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.823200 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec with delete_type TABLET_DATA_DELETED (replaced by tablet 6049ed1c2d84494893bfa20a9d563838) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.825256 28736 tablet_bootstrap.cc:492] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.825429 28736 ts_tablet_manager.cc:1403] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.001s
I20260320 15:27:48.826467 28736 raft_consensus.cc:359] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.826710 28736 raft_consensus.cc:385] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.826762 28736 raft_consensus.cc:740] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.828747 28736 consensus_queue.cc:260] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.829708 28901 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 806d635bb5594f0bb826b9b31468b485 consensus state has no leader
I20260320 15:27:48.830148 28736 ts_tablet_manager.cc:1434] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:48.830849 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:48.831352 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet db16d86664794305bde714a7ff4d5caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.831836 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a3d5e4ff8f42409482af49c636be0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.834496 28736 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.836606 28736 tablet_bootstrap.cc:654] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:48.838214 28900 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 111) took 2368 ms (client timeout 1000 ms). Trace:
I20260320 15:27:48.838766 28733 tablet_bootstrap.cc:492] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.838975 28733 ts_tablet_manager.cc:1403] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.152s user 0.006s sys 0.010s
I20260320 15:27:48.839524 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d92e17cea044541968356ae993be1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:33 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.840004 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 35347116811e4aee8142a503dd21843a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.840837 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.838392 28900 rpcz_store.cc:269] 0320 15:27:46.469542 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.473969 (+ 4427us) service_pool.cc:225] Handling call
0320 15:27:46.496415 (+ 22446us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf1448 00007fe6118e2ca6 00007fe6118e29d5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:48.838141 (+2341726us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:47.961283 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:47.961585 (+ 302us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:47.961596 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:47.961600 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:47.961750 (+ 150us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:47.961772 (+ 22us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:47.961780 (+ 8us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:47.961873 (+ 93us) tablet.cc:700] Row locks acquired
0320 15:27:47.961877 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:47.961933 (+ 56us) write_op.cc:270] Start()
0320 15:27:47.961980 (+ 47us) write_op.cc:276] Timestamp: P: 1774020467961899 usec, L: 0
0320 15:27:47.961983 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:47.962226 (+ 243us) log.cc:844] Serialized 527 byte log entry
0320 15:27:47.978387 (+ 16161us) op_driver.cc:464] REPLICATION: finished
0320 15:27:47.978545 (+ 158us) write_op.cc:301] APPLY: starting
0320 15:27:47.978573 (+ 28us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:47.978639 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:47.978693 (+ 54us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:47.978785 (+ 92us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:47.978839 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:47.978851 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:47.978964 (+ 113us) log.cc:844] Serialized 23 byte log entry
0320 15:27:47.979074 (+ 110us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:47.979107 (+ 33us) write_op.cc:454] Released schema lock
0320 15:27:47.979117 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:47.979199 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:47.979203 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":60451,"reported_tablets":168,"spinlock_wait_cycles":42582400,"child_traces":[["op",{"apply.queue_time_us":86,"apply.run_cpu_time_us":648,"apply.run_wall_time_us":648,"num_ops":1,"prepare.queue_time_us":800,"prepare.run_cpu_time_us":1112,"prepare.run_wall_time_us":1113,"raft.queue_time_us":1363,"raft.run_cpu_time_us":576,"raft.run_wall_time_us":575,"replication_time_us":16348}]]}
I20260320 15:27:48.841629 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet ac4c7dc6859048ce956927201df44a86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.842283 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_1c80a258a5e34d8b9708d39f47c0fdd8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:48.842852 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.843255 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet cc97a5f2bd964a05bb7b2427226debdd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a2003e2af9a4d9c854990e1db41af28) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.843786 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet fcff253b9cce45b4ad11dc38397afee2 with delete_type TABLET_DATA_DELETED (replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.844201 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet b4cad900411e4c9686701aa99206dd23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.844579 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.845016 28272 tablet_service.cc:1558] Processing DeleteTablet for tablet 28c453eaf49c4171b0f93d3b26690d83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.845240 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:43 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.845589 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 8241a342054240d2ac62b729cc9c9a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:25 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.845939 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 56fa691a26114657ad768545a30dfd3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.846256 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet dd315b5d49104d679b03515b65ee30ac with delete_type TABLET_DATA_DELETED (replaced by tablet ef0d80754dfd4bbd9817f568063511ef) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.846848 28905 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 110) took 3522 ms (client timeout 1000 ms). Trace:
I20260320 15:27:48.846889 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.849269 28889 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.2:50195 (request call id 112) took 1344 ms (client timeout 1000 ms). Trace:
W20260320 15:27:48.849442 28889 rpcz_store.cc:269] 0320 15:27:47.505001 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.953038 (+448037us) service_pool.cc:225] Handling call
0320 15:27:48.392496 (+439458us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.392505 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:48.849212 (+456707us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.382104 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.382245 (+ 141us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.382252 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:48.382255 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.382462 (+ 207us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.382478 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.382483 (+ 5us) tablet.cc:684] Acquiring locks for 3 operations
0320 15:27:48.382574 (+ 91us) tablet.cc:700] Row locks acquired
0320 15:27:48.382577 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:48.382615 (+ 38us) write_op.cc:270] Start()
0320 15:27:48.382653 (+ 38us) write_op.cc:276] Timestamp: P: 1774020468382589 usec, L: 0
0320 15:27:48.382656 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.382896 (+ 240us) log.cc:844] Serialized 1275 byte log entry
0320 15:27:48.387435 (+ 4539us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.387726 (+ 291us) write_op.cc:301] APPLY: starting
0320 15:27:48.387751 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.387820 (+ 69us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.387866 (+ 46us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.387981 (+ 115us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.388030 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0320 15:27:48.388041 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:48.388153 (+ 112us) log.cc:844] Serialized 35 byte log entry
0320 15:27:48.388252 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.388286 (+ 34us) write_op.cc:454] Released schema lock
0320 15:27:48.388296 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:48.712522 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.712665 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.712673 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:48.712676 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.712791 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.712810 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.712816 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:48.712890 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:48.712894 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:48.712932 (+ 38us) write_op.cc:270] Start()
0320 15:27:48.712973 (+ 41us) write_op.cc:276] Timestamp: P: 1774020468712877 usec, L: 0
0320 15:27:48.712977 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.713255 (+ 278us) log.cc:844] Serialized 356 byte log entry
0320 15:27:48.728592 (+ 15337us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.728766 (+ 174us) write_op.cc:301] APPLY: starting
0320 15:27:48.728788 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.728836 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.728866 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.728938 (+ 72us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.728977 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.728985 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:48.729102 (+ 117us) log.cc:844] Serialized 23 byte log entry
0320 15:27:48.729192 (+ 90us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.729230 (+ 38us) write_op.cc:454] Released schema lock
0320 15:27:48.729244 (+ 14us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.729369 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.729376 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79327,"reported_tablets":170,"child_traces":[["op",{"apply.queue_time_us":200,"apply.run_cpu_time_us":654,"apply.run_wall_time_us":653,"num_ops":3,"prepare.queue_time_us":3892,"prepare.run_cpu_time_us":923,"prepare.run_wall_time_us":923,"raft.queue_time_us":302,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":367,"replication_time_us":4743}],["op",{"apply.queue_time_us":102,"apply.run_cpu_time_us":587,"apply.run_wall_time_us":585,"num_ops":1,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":886,"prepare.run_wall_time_us":885,"raft.queue_time_us":129,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":355,"replication_time_us":15564}]]}
I20260320 15:27:48.850679 30841 tablet.cc:1723] T 408f5b4d67514838a17c180ae2cd71df P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:48.850955 28889 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8aca34fa920c499eb7e2ae000ced9d02" } modify_external_catalogs: true
W20260320 15:27:48.853477 28901 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 126) took 1728 ms (client timeout 1000 ms). Trace:
W20260320 15:27:48.853637 28901 rpcz_store.cc:269] 0320 15:27:47.124925 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.948767 (+823842us) service_pool.cc:225] Handling call
0320 15:27:48.853393 (+904626us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":87063,"reported_tablets":178}
I20260320 15:27:48.853973 28901 master_service.cc:954] ReplaceTablet: received request to replace tablet 5de834725bda4f249882914f1247ab62 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:48.855108 28733 raft_consensus.cc:359] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.855470 28733 raft_consensus.cc:385] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.855647 28733 raft_consensus.cc:740] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.855973 28733 consensus_queue.cc:260] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.856683 28733 ts_tablet_manager.cc:1434] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.018s user 0.000s sys 0.003s
W20260320 15:27:48.856837 28890 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 124) took 4014 ms (client timeout 1000 ms). Trace:
I20260320 15:27:48.857519 28733 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:48.856971 28890 rpcz_store.cc:269] 0320 15:27:44.842024 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.991984 (+149960us) service_pool.cc:225] Handling call
0320 15:27:48.856764 (+3864780us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.682787 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.682935 (+ 148us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.682945 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:45.682949 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:45.683097 (+ 148us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.683115 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.683120 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:45.683237 (+ 117us) tablet.cc:700] Row locks acquired
0320 15:27:45.683242 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:45.683298 (+ 56us) write_op.cc:270] Start()
0320 15:27:45.683333 (+ 35us) write_op.cc:276] Timestamp: P: 1774020465683243 usec, L: 0
0320 15:27:45.683337 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.683651 (+ 314us) log.cc:844] Serialized 901 byte log entry
0320 15:27:45.687406 (+ 3755us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.687564 (+ 158us) write_op.cc:301] APPLY: starting
0320 15:27:45.687587 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.687641 (+ 54us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.687680 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.687778 (+ 98us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.687837 (+ 59us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:45.687844 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.687937 (+ 93us) log.cc:844] Serialized 29 byte log entry
0320 15:27:45.688007 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.688032 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:45.688040 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:45.688115 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.688119 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64868,"reported_tablets":178,"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":541,"apply.run_wall_time_us":541,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":899,"prepare.run_wall_time_us":966,"raft.queue_time_us":398,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":282,"replication_time_us":4034}]]}
I20260320 15:27:48.857992 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c9ff393f07814790b10d6c0faef3d408"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:48.858681 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9ff393f07814790b10d6c0faef3d408 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:48.820873 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet e779a274913f4caeaf82e3f180457830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.820931 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 48c1002cbdd845e2b96dcb57c450cc6d with delete_type TABLET_DATA_DELETED (replaced by tablet 740981a8159243088548b90b0c576028) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:48.859707 28239 consensus_peers.cc:597] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:48.859813 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 90cd18e2ccbf48c8b804329bd3abf847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.821022 28424 tablet_service.cc:1558] Processing DeleteTablet for tablet a185f84c24ea402684aa1cc3628d4f11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.821981 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec with delete_type TABLET_DATA_DELETED (replaced by tablet 6049ed1c2d84494893bfa20a9d563838) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.822063 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 20eec704800e4d8b924d97556cc2700e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:37 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.860035 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 197f1935099140a98c5eb801cf44046a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:31 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.860138 28421 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f94140a5e8b4a06890bca86456d64a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.860163 28422 tablet_service.cc:1558] Processing DeleteTablet for tablet 06cb44b9cc6445cf8324cf52190d6773 with delete_type TABLET_DATA_DELETED (replaced by tablet 91f393d2ca304b899f788add43da7e9a) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.821869 28411 tablet_service.cc:1558] Processing DeleteTablet for tablet 4190329df1b644c392ff8cbace9aae7a with delete_type TABLET_DATA_DELETED (replaced by tablet 806d635bb5594f0bb826b9b31468b485) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825099 28423 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 with delete_type TABLET_DATA_DELETED (replaced by tablet aaefdda7483e4c21b09b7c868395c6f1) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825223 28420 tablet_service.cc:1558] Processing DeleteTablet for tablet aba79ddde6e842dfb67597c5d9e371cb with delete_type TABLET_DATA_DELETED (replaced by tablet 71db28879e554f578b8a7b0e14419ab7) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.860051 28426 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bd3032950254e6f9db8da35dbb1acc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:43 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825676 28413 tablet_service.cc:1558] Processing DeleteTablet for tablet d823ab41194542fba5540e53e9f4680b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825812 28408 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a4b5a658cd84b04b50b54973c065889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825906 28410 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.825999 28418 tablet_service.cc:1558] Processing DeleteTablet for tablet 54266dd376164673bffe829dd515b83d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:30 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.826117 28419 tablet_service.cc:1558] Processing DeleteTablet for tablet e2243e6811744b7cba99a487f8e8bb83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.826210 28414 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.826778 28427 tablet_service.cc:1558] Processing DeleteTablet for tablet b16ffb28029e4b7ca37ce36efe9e0271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.829360 28412 tablet_service.cc:1558] Processing DeleteTablet for tablet db16d86664794305bde714a7ff4d5caf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.829515 28415 tablet_service.cc:1558] Processing DeleteTablet for tablet 87a3d5e4ff8f42409482af49c636be0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:44 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.829645 28417 tablet_service.cc:1558] Processing DeleteTablet for tablet cc97a5f2bd964a05bb7b2427226debdd with delete_type TABLET_DATA_DELETED (replaced by tablet 9a2003e2af9a4d9c854990e1db41af28) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.838167 30687 raft_consensus.cc:493] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.839418 29643 ts_tablet_manager.cc:1929] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:48.860981 29643 log.cc:1199] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/49654c67435a48448794879464f6cfb2
I20260320 15:27:48.860966 30687 raft_consensus.cc:515] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.861515 29643 ts_tablet_manager.cc:1950] T 49654c67435a48448794879464f6cfb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.861766 30687 leader_election.cc:290] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.840694 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d9b0f1936f54c699211c5ee9709f9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:26 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.862437 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 388) took 2247 ms. Trace:
I20260320 15:27:48.862480 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.862520 29643 rpcz_store.cc:276] 0320 15:27:46.615147 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.615246 (+ 99us) service_pool.cc:225] Handling call
0320 15:27:48.765567 (+2150321us) ts_tablet_manager.cc:1151] Deleting tablet 49654c67435a48448794879464f6cfb2
0320 15:27:48.765584 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.788304 (+ 22720us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.839284 (+ 50980us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.862418 (+ 23134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2150014}
I20260320 15:27:48.847712 30456 raft_consensus.cc:493] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.862848 30456 raft_consensus.cc:515] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.862972 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 387) took 2248 ms. Trace:
I20260320 15:27:48.863035 29643 rpcz_store.cc:276] 0320 15:27:46.614925 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.615972 (+ 1047us) service_pool.cc:225] Handling call
0320 15:27:48.862812 (+2246840us) ts_tablet_manager.cc:1151] Deleting tablet 7840b529206d482283cc550d2c5e7606
0320 15:27:48.862827 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.862956 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2246644}
I20260320 15:27:48.863296 29643 tablet_replica.cc:333] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:48.863471 30456 leader_election.cc:290] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.863981 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 87cb35cefee841d0881b08c456df5a97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.864084 29643 raft_consensus.cc:2243] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:48.864151 28382 leader_election.cc:343] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.864319 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 35347116811e4aee8142a503dd21843a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:40 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.864423 29643 raft_consensus.cc:2272] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.864395 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.864650 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 2e9791ac61b34fc09a8292c79f0c4032 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:41 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:48.864908 28382 leader_election.cc:343] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.864997 29643 ts_tablet_manager.cc:1916] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.865113 28417 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.859813 28409 tablet_service.cc:1558] Processing DeleteTablet for tablet b4cad900411e4c9686701aa99206dd23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:45 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.824949 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 27ccc693e7704650bd77e0c4ca11862e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:34 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.864928 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.865767 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ed5d220ec4b406aa5bae970f785de63 with delete_type TABLET_DATA_DELETED (replaced by tablet bf37dbb8bc8e48c5a11f106ecd93dd78) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.866034 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.866189 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet fcff253b9cce45b4ad11dc38397afee2 with delete_type TABLET_DATA_DELETED (replaced by tablet a8b40dfe348b48ea8d941e3d97f27d73) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.866365 30395 raft_consensus.cc:493] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.866495 30395 raft_consensus.cc:515] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.867411 30395 leader_election.cc:290] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:48.867477 28906 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.1:41749 (request call id 125) took 2870 ms (client timeout 1000 ms). Trace:
I20260320 15:27:48.867946 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.868366 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.868491 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 6049ed1c2d84494893bfa20a9d563838 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.868114 28906 rpcz_store.cc:269] 0320 15:27:45.996816 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.173875 (+177059us) service_pool.cc:225] Handling call
0320 15:27:48.867387 (+2693512us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:47.559398 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:47.559600 (+ 202us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:47.559608 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:47.559611 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:47.559769 (+ 158us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:47.559787 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:47.559792 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:47.559883 (+ 91us) tablet.cc:700] Row locks acquired
0320 15:27:47.559886 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:47.559957 (+ 71us) write_op.cc:270] Start()
0320 15:27:47.560003 (+ 46us) write_op.cc:276] Timestamp: P: 1774020467559910 usec, L: 0
0320 15:27:47.560008 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:47.560264 (+ 256us) log.cc:844] Serialized 527 byte log entry
0320 15:27:47.620626 (+ 60362us) op_driver.cc:464] REPLICATION: finished
0320 15:27:47.628065 (+ 7439us) write_op.cc:301] APPLY: starting
0320 15:27:47.628086 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:47.628135 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:47.628184 (+ 49us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:47.628258 (+ 74us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:47.628297 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:47.628304 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:47.628443 (+ 139us) log.cc:844] Serialized 23 byte log entry
0320 15:27:47.628524 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:47.628555 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:47.628561 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":280611,"reported_tablets":168,"child_traces":[["op",{"apply.queue_time_us":7340,"apply.run_cpu_time_us":585,"apply.run_wall_time_us":585,"num_ops":1,"prepare.queue_time_us":16782,"prepare.run_cpu_time_us":1130,"prepare.run_wall_time_us":4388,"raft.queue_time_us":1482,"raft.run_cpu_time_us":734,"raft.run_wall_time_us":734,"replication_time_us":60553,"thread_start_us":118144,"threads_started":2}]]}
I20260320 15:27:48.868763 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.868834 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 71ec63fb080545de838803033f204fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:27 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.868971 28437 raft_consensus.cc:2468] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:48.869136 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 6292904782fd4b619483135e58b66684 with delete_type TABLET_DATA_DELETED (replaced by tablet 92ea5e924a734c8ba6dfb641359593a3) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.869297 30719 raft_consensus.cc:493] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.869253 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_7b38daf0b39d41f18e3f8a65deca4870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:48.869416 30719 raft_consensus.cc:515] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.869493 28524 leader_election.cc:304] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.870080 30719 leader_election.cc:290] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:48.870097 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b38daf0b39d41f18e3f8a65deca4870 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:48.870735 30395 raft_consensus.cc:2804] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.870757 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5618796f1a4ff3ab4c1209dad0aede with delete_type TABLET_DATA_DELETED (replaced by tablet 3f4a2c8ebc374de9bd8078a757c6be57) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.870827 30395 raft_consensus.cc:493] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.870894 30395 raft_consensus.cc:3060] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.871132 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 16e31e11048b494ebe1fd4d89488b0cf with delete_type TABLET_DATA_DELETED (replaced by tablet 547d82cee19b4959b3a779c8b8088e83) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.871428 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 5a20d7f22cfa403184e73b24d565b60a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:35 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.871745 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:46760
W20260320 15:27:48.872087 28383 leader_election.cc:343] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.872211 28383 leader_election.cc:304] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.872565 28736 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.872627 30456 raft_consensus.cc:2749] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.872713 28736 ts_tablet_manager.cc:1403] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.000s
I20260320 15:27:48.873390 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20eec704800e4d8b924d97556cc2700e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.873582 28437 raft_consensus.cc:2468] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.873550 28736 raft_consensus.cc:359] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.873710 28736 raft_consensus.cc:385] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.873770 28736 raft_consensus.cc:740] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.873962 28736 consensus_queue.cc:260] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.874065 28240 leader_election.cc:304] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:48.874203 28383 leader_election.cc:343] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.874320 28383 leader_election.cc:304] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.874369 30719 raft_consensus.cc:2804] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.874431 30719 raft_consensus.cc:493] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.874490 30719 raft_consensus.cc:3060] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.874660 28736 ts_tablet_manager.cc:1434] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:48.874714 30456 raft_consensus.cc:2749] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:48.874945 28736 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.876689 28736 tablet_bootstrap.cc:654] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.879520 30719 raft_consensus.cc:515] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.880090 30840 tablet_replica.cc:333] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:48.880323 30840 raft_consensus.cc:2243] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:48.880550 30840 pending_rounds.cc:70] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:48.880600 30840 pending_rounds.cc:77] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387839366610944 op_type: NO_OP noop_request { }
I20260320 15:27:48.880671 30719 leader_election.cc:290] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:48.880687 30840 raft_consensus.cc:2889] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:48.880766 30840 raft_consensus.cc:2272] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:48.880937 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20eec704800e4d8b924d97556cc2700e" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.881107 28437 raft_consensus.cc:3060] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.884429 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.884708 28294 raft_consensus.cc:2468] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
W20260320 15:27:48.885142 28239 leader_election.cc:343] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 20eec704800e4d8b924d97556cc2700e
W20260320 15:27:48.885603 28239 leader_election.cc:343] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Not found: Tablet not found: 20eec704800e4d8b924d97556cc2700e
I20260320 15:27:48.885602 30395 raft_consensus.cc:515] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.886361 30395 leader_election.cc:290] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.888957 28733 tablet_bootstrap.cc:654] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.889600 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.889889 28439 raft_consensus.cc:3060] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.890503 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "813604f6c6c747eda18f7a01d9d8ec97" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:48.890730 28294 raft_consensus.cc:3060] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.894075 28294 raft_consensus.cc:2468] T 813604f6c6c747eda18f7a01d9d8ec97 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:48.895860 28524 leader_election.cc:304] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
W20260320 15:27:48.897481 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 because the tablet was not found. No further retry: Not found: Tablet not found: 41da3cb06b1147fbae2695bc5c9ffaa5
W20260320 15:27:48.897974 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8428019c3b074b17aec493c14cc2b911 because the tablet was not found. No further retry: Not found: Tablet not found: 8428019c3b074b17aec493c14cc2b911
I20260320 15:27:48.898168 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 49654c67435a48448794879464f6cfb2 (table default.Table_32e1ad85cf09464a87ffd6feb70093cc [id=ba3a0a1baa0b422fb73f660743dc168a]) successfully deleted
W20260320 15:27:48.898406 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7840b529206d482283cc550d2c5e7606 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7840b529206d482283cc550d2c5e7606 already in progress: creating tablet
I20260320 15:27:48.898861 28439 raft_consensus.cc:2468] T 813604f6c6c747eda18f7a01d9d8ec97 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
W20260320 15:27:48.899641 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c80a258a5e34d8b9708d39f47c0fdd8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:48.900123 30630 raft_consensus.cc:2804] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.900293 30630 raft_consensus.cc:697] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:48.900593 30630 consensus_queue.cc:237] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.901077 30719 raft_consensus.cc:493] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.901221 30719 raft_consensus.cc:515] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.902400 30719 leader_election.cc:290] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:48.847007 28905 rpcz_store.cc:269] 0320 15:27:45.324772 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.337387 (+ 12615us) service_pool.cc:225] Handling call
0320 15:27:45.921419 (+584032us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.921429 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:48.846797 (+2925368us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:45.911396 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:45.911561 (+ 165us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:45.911571 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:45.911574 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:45.911698 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:45.911716 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:45.911721 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:45.911794 (+ 73us) tablet.cc:700] Row locks acquired
0320 15:27:45.911799 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:45.911848 (+ 49us) write_op.cc:270] Start()
0320 15:27:45.911895 (+ 47us) write_op.cc:276] Timestamp: P: 1774020465911812 usec, L: 0
0320 15:27:45.911899 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:45.912125 (+ 226us) log.cc:844] Serialized 527 byte log entry
0320 15:27:45.916763 (+ 4638us) op_driver.cc:464] REPLICATION: finished
0320 15:27:45.916899 (+ 136us) write_op.cc:301] APPLY: starting
0320 15:27:45.916918 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:45.916968 (+ 50us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:45.917003 (+ 35us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:45.917072 (+ 69us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:45.917111 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:45.917118 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:45.917207 (+ 89us) log.cc:844] Serialized 23 byte log entry
0320 15:27:45.917287 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:45.917313 (+ 26us) write_op.cc:454] Released schema lock
0320 15:27:45.917320 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0320 15:27:48.056331 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.056471 (+ 140us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.056481 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:48.056485 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:48.056615 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.056629 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.056634 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:48.056706 (+ 72us) tablet.cc:700] Row locks acquired
0320 15:27:48.056709 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:48.056750 (+ 41us) write_op.cc:270] Start()
0320 15:27:48.056790 (+ 40us) write_op.cc:276] Timestamp: P: 1774020468056713 usec, L: 0
0320 15:27:48.056793 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.057017 (+ 224us) log.cc:844] Serialized 356 byte log entry
0320 15:27:48.068655 (+ 11638us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.069173 (+ 518us) write_op.cc:301] APPLY: starting
0320 15:27:48.069195 (+ 22us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.069243 (+ 48us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.071114 (+ 1871us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.071216 (+ 102us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.071272 (+ 56us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.071284 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:48.071415 (+ 131us) log.cc:844] Serialized 23 byte log entry
0320 15:27:48.071556 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.071579 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:48.071586 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.071697 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.071703 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":72521,"reported_tablets":170,"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":479,"num_ops":1,"prepare.queue_time_us":1100,"prepare.run_cpu_time_us":863,"prepare.run_wall_time_us":863,"raft.queue_time_us":140,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":292,"replication_time_us":4822}],["op",{"apply.queue_time_us":445,"apply.run_cpu_time_us":718,"apply.run_wall_time_us":2508,"num_ops":1,"prepare.queue_time_us":632,"prepare.run_cpu_time_us":788,"prepare.run_wall_time_us":788,"replication_time_us":11828}]]}
I20260320 15:27:48.903257 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.903571 28905 master_service.cc:954] ReplaceTablet: received request to replace tablet 49654c67435a48448794879464f6cfb2 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:48.903702 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:48.903882 28444 raft_consensus.cc:2468] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.904328 28240 leader_election.cc:304] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.904624 30773 raft_consensus.cc:493] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.904752 30773 raft_consensus.cc:515] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.905352 30773 leader_election.cc:290] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.905501 30773 raft_consensus.cc:2804] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.905565 30773 raft_consensus.cc:493] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.905646 30773 raft_consensus.cc:3060] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.905727 28239 leader_election.cc:343] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.906508 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed5d220ec4b406aa5bae970f785de63" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.906689 28582 raft_consensus.cc:2468] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:48.907063 28239 leader_election.cc:304] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:48.907024 30466 consensus_queue.cc:1048] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.907498 30466 raft_consensus.cc:2804] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:48.907600 30466 raft_consensus.cc:493] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:48.907672 30466 raft_consensus.cc:3060] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:48.908921 28240 leader_election.cc:343] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:48.910408 30773 raft_consensus.cc:515] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.910533 30466 raft_consensus.cc:515] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.911152 30466 leader_election.cc:290] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:48.911489 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 221) took 8578 ms. Trace:
I20260320 15:27:48.911618 29640 rpcz_store.cc:276] 0320 15:27:40.332788 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.334605 (+ 1817us) service_pool.cc:225] Handling call
0320 15:27:48.585048 (+8250443us) ts_tablet_manager.cc:1151] Deleting tablet 9dbd3b0e92a744dcbc132037c67303ab
0320 15:27:48.585058 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.666500 (+ 81442us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.769434 (+102934us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.911433 (+141999us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8250191}
I20260320 15:27:48.912113 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 9dbd3b0e92a744dcbc132037c67303ab (table default.Table_6d5fc724ebb540cbb35c767edfe19833 [id=634e24fdc81640fbb393a4b19bac7343]) successfully deleted
I20260320 15:27:48.912343 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:48.912552 28444 raft_consensus.cc:3060] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.913542 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 222) took 8572 ms. Trace:
I20260320 15:27:48.913776 29640 rpcz_store.cc:276] 0320 15:27:40.341542 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.341675 (+ 133us) service_pool.cc:225] Handling call
0320 15:27:48.912847 (+8571172us) ts_tablet_manager.cc:1151] Deleting tablet b1bc86d3d1ff4b3aa8463eb21f619917
0320 15:27:48.912869 (+ 22us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.913441 (+ 572us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8570785}
W20260320 15:27:48.915016 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b1bc86d3d1ff4b3aa8463eb21f619917 because the tablet was not found. No further retry: Not found: Tablet not found: b1bc86d3d1ff4b3aa8463eb21f619917
I20260320 15:27:48.921051 28892 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 164) took 2360 ms. Trace:
I20260320 15:27:48.921214 28892 rpcz_store.cc:276] 0320 15:27:46.560315 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.096830 (+1536515us) service_pool.cc:225] Handling call
0320 15:27:48.120381 (+ 23551us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:48.120662 (+ 281us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:48.120928 (+ 266us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:48.121029 (+ 101us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:48.187206 (+ 66177us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:48.920953 (+733747us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.177049 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.177229 (+ 180us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.177236 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:48.177239 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.177372 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.177389 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.177395 (+ 6us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:48.177469 (+ 74us) tablet.cc:700] Row locks acquired
0320 15:27:48.177472 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:48.177518 (+ 46us) write_op.cc:270] Start()
0320 15:27:48.177555 (+ 37us) write_op.cc:276] Timestamp: P: 1774020468177466 usec, L: 0
0320 15:27:48.177559 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.177793 (+ 234us) log.cc:844] Serialized 405 byte log entry
0320 15:27:48.186197 (+ 8404us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.186326 (+ 129us) write_op.cc:301] APPLY: starting
0320 15:27:48.186346 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.186393 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.186431 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.186494 (+ 63us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.186548 (+ 54us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.186557 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:48.186653 (+ 96us) log.cc:844] Serialized 23 byte log entry
0320 15:27:48.186748 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.186771 (+ 23us) write_op.cc:454] Released schema lock
0320 15:27:48.186780 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.187059 (+ 279us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.187069 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22216,"child_traces":[["op",{"apply.queue_time_us":75,"apply.run_cpu_time_us":534,"apply.run_wall_time_us":533,"num_ops":1,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":839,"prepare.run_wall_time_us":838,"replication_time_us":8602}]]}
I20260320 15:27:48.923223 28736 tablet_bootstrap.cc:492] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.923386 28736 ts_tablet_manager.cc:1403] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.000s
I20260320 15:27:48.924188 28736 raft_consensus.cc:359] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.924391 28736 raft_consensus.cc:385] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.924444 28736 raft_consensus.cc:740] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.924641 28736 consensus_queue.cc:260] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.925036 28736 ts_tablet_manager.cc:1434] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260320 15:27:48.925184 28240 leader_election.cc:343] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:48.925335 28736 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.926962 28736 tablet_bootstrap.cc:654] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.928809 28444 raft_consensus.cc:2468] T 8eb3d5e19cda48fda124b6238902c03a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.929596 28240 leader_election.cc:304] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:48.931509 30773 leader_election.cc:290] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:48.931740 30773 raft_consensus.cc:2804] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.931881 30773 raft_consensus.cc:697] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.937111 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.939023 28437 raft_consensus.cc:2468] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.939759 28240 leader_election.cc:304] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:48.940748 30719 raft_consensus.cc:2804] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.940909 30719 raft_consensus.cc:697] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.941206 30719 consensus_queue.cc:237] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.941758 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ed5d220ec4b406aa5bae970f785de63" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:48.941978 28575 raft_consensus.cc:3060] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:48.932238 30773 consensus_queue.cc:237] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:48.945116 28239 leader_election.cc:343] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.945861 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 5de834725bda4f249882914f1247ab62
I20260320 15:27:48.947163 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de834725bda4f249882914f1247ab62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b527973ede040e286f7b8f2e7536699) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:48.947829 28901 catalog_manager.cc:6741] ReplaceTablet: tablet 5de834725bda4f249882914f1247ab62 deleted and replaced by tablet 4b527973ede040e286f7b8f2e7536699
I20260320 15:27:48.948179 28901 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 168) took 2355 ms. Trace:
I20260320 15:27:48.948412 28901 rpcz_store.cc:276] 0320 15:27:46.593147 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.853874 (+2260727us) service_pool.cc:225] Handling call
0320 15:27:48.945410 (+ 91536us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.945432 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:48.948129 (+ 2697us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.936763 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.936909 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.936918 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:48.936921 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.937044 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.937063 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.937069 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:48.937161 (+ 92us) tablet.cc:700] Row locks acquired
0320 15:27:48.937164 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:48.937207 (+ 43us) write_op.cc:270] Start()
0320 15:27:48.937249 (+ 42us) write_op.cc:276] Timestamp: P: 1774020468937179 usec, L: 0
0320 15:27:48.937254 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.937483 (+ 229us) log.cc:844] Serialized 656 byte log entry
0320 15:27:48.941720 (+ 4237us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.941845 (+ 125us) write_op.cc:301] APPLY: starting
0320 15:27:48.941868 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.941922 (+ 54us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.941959 (+ 37us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.942107 (+ 148us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.942191 (+ 84us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:48.942200 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:48.942297 (+ 97us) log.cc:844] Serialized 29 byte log entry
0320 15:27:48.942373 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.942398 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:48.942407 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":635,"apply.run_wall_time_us":634,"num_ops":2,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":844,"prepare.run_wall_time_us":844,"raft.queue_time_us":81,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":373,"replication_time_us":4424}]]}
W20260320 15:27:48.950538 28495 tablet.cc:2379] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:48.949688 28889 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8aca34fa920c499eb7e2ae000ced9d02" } modify_external_catalogs: true
I20260320 15:27:48.953362 30466 consensus_queue.cc:1048] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:48.955015 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de834725bda4f249882914f1247ab62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b527973ede040e286f7b8f2e7536699) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:48.958767 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de834725bda4f249882914f1247ab62 with delete_type TABLET_DATA_DELETED (replaced by tablet 4b527973ede040e286f7b8f2e7536699) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:48.964479 28736 tablet_bootstrap.cc:492] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:48.964659 28736 ts_tablet_manager.cc:1403] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.000s
I20260320 15:27:48.965345 30840 ts_tablet_manager.cc:1916] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:48.968444 30842 tablet.cc:1723] T d78ab2f7e0d7413ea9049dba9cbd8633 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:48.969033 30849 tablet.cc:1723] T d78ab2f7e0d7413ea9049dba9cbd8633 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:48.965410 28736 raft_consensus.cc:359] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.973152 28736 raft_consensus.cc:385] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:48.973533 28736 raft_consensus.cc:740] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:48.973905 28736 consensus_queue.cc:260] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.966775 30848 tablet.cc:1723] T d78ab2f7e0d7413ea9049dba9cbd8633 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260320 15:27:48.974663 28736 ts_tablet_manager.cc:1434] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.010s user 0.002s sys 0.001s
I20260320 15:27:48.975069 28736 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:48.977063 28736 tablet_bootstrap.cc:654] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:48.981967 28575 raft_consensus.cc:2468] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:48.982956 29643 ts_tablet_manager.cc:1929] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.984198 30456 raft_consensus.cc:493] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:48.985896 30456 raft_consensus.cc:515] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.986097 28239 leader_election.cc:304] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:48.986428 30773 raft_consensus.cc:2804] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:48.986585 30773 raft_consensus.cc:697] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:48.986689 30456 leader_election.cc:290] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:48.986842 30773 consensus_queue.cc:237] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:48.987466 28733 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:48.987406 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:48.987942 29643 log.cc:1199] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/4cfad5cf843a437e8e15c7236d1dc5fe
W20260320 15:27:48.988098 28382 leader_election.cc:343] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.988634 29643 ts_tablet_manager.cc:1950] T 4cfad5cf843a437e8e15c7236d1dc5fe P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:48.989715 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 389) took 1048 ms. Trace:
I20260320 15:27:48.989836 29643 rpcz_store.cc:276] 0320 15:27:47.940832 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.940916 (+ 84us) service_pool.cc:225] Handling call
0320 15:27:48.863255 (+922339us) ts_tablet_manager.cc:1151] Deleting tablet 4cfad5cf843a437e8e15c7236d1dc5fe
0320 15:27:48.863263 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.975892 (+112629us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.982832 (+ 6940us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:48.989685 (+ 6853us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":922060}
I20260320 15:27:48.993278 30840 ts_tablet_manager.cc:1929] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:48.993427 30840 log.cc:1199] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/48c1002cbdd845e2b96dcb57c450cc6d
I20260320 15:27:48.996899 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:48.997152 28294 raft_consensus.cc:2468] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:48.998308 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 223) took 8633 ms. Trace:
I20260320 15:27:48.999225 28733 ts_tablet_manager.cc:1403] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.142s user 0.001s sys 0.008s
I20260320 15:27:49.000015 28383 leader_election.cc:304] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.000416 30456 raft_consensus.cc:2804] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.000499 30456 raft_consensus.cc:493] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.000555 30456 raft_consensus.cc:3060] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.000615 28734 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.000763 28734 ts_tablet_manager.cc:1403] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.211s user 0.002s sys 0.009s
I20260320 15:27:49.002081 30719 raft_consensus.cc:493] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.002239 30719 raft_consensus.cc:515] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.002879 30719 leader_election.cc:290] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.003569 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.004807 28294 raft_consensus.cc:3060] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.008469 30862 tablet.cc:1723] T 408f5b4d67514838a17c180ae2cd71df P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:49.009672 30395 consensus_queue.cc:1048] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.009953 30456 raft_consensus.cc:515] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.010007 28733 raft_consensus.cc:359] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.010344 28733 raft_consensus.cc:385] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.010424 28733 raft_consensus.cc:740] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.010879 28733 consensus_queue.cc:260] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.011176 30456 leader_election.cc:290] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.012369 28733 ts_tablet_manager.cc:1434] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260320 15:27:49.012707 28733 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.012688 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:49.012856 28294 raft_consensus.cc:3060] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.012826 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:49.013078 28736 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.013209 28736 ts_tablet_manager.cc:1403] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.000s
W20260320 15:27:49.013412 28382 leader_election.cc:343] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:48.998456 29640 rpcz_store.cc:276] 0320 15:27:40.364851 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.364919 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:48.916043 (+8551124us) ts_tablet_manager.cc:1151] Deleting tablet 591ccc1d349e4fd599d5e343f91defe2
0320 15:27:48.916125 (+ 82us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:48.998271 (+ 82146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8550727}
I20260320 15:27:49.014605 29640 tablet_replica.cc:333] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.014858 29640 raft_consensus.cc:2243] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.015301 29640 pending_rounds.cc:70] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Trying to abort 1 pending ops.
W20260320 15:27:49.017782 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 591ccc1d349e4fd599d5e343f91defe2 because the tablet was not found. No further retry: Not found: Tablet not found: 591ccc1d349e4fd599d5e343f91defe2
I20260320 15:27:49.019448 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 4cfad5cf843a437e8e15c7236d1dc5fe (table default.Table_b35323a629d64449b3f893c3049761b6 [id=30ce4e521c3c4372b3b9ccba75714597]) successfully deleted
I20260320 15:27:49.015136 28736 raft_consensus.cc:359] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.020120 28736 raft_consensus.cc:385] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.020213 28736 raft_consensus.cc:740] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.020539 28736 consensus_queue.cc:260] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.021271 28736 ts_tablet_manager.cc:1434] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260320 15:27:49.021695 28736 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.022603 28905 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 171) took 2313 ms. Trace:
I20260320 15:27:49.023741 28294 raft_consensus.cc:2468] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:49.026331 28383 leader_election.cc:304] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.026782 30687 raft_consensus.cc:2804] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.026934 30687 raft_consensus.cc:697] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:49.027212 30687 consensus_queue.cc:237] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.025390 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da29d2ffcb07428789c22a573d680289" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.035372 29643 tablet_replica.cc:333] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.036156 30840 ts_tablet_manager.cc:1950] T 48c1002cbdd845e2b96dcb57c450cc6d P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.035497 28447 raft_consensus.cc:2393] T da29d2ffcb07428789c22a573d680289 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:49.037158 28240 leader_election.cc:304] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.037930 30719 raft_consensus.cc:2749] T da29d2ffcb07428789c22a573d680289 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.038601 29643 raft_consensus.cc:2243] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.038787 29643 raft_consensus.cc:2272] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.039371 29643 ts_tablet_manager.cc:1916] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.039963 28901 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_39290b96eadb43968da86872c4682a5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:49.040867 28905 rpcz_store.cc:276] 0320 15:27:46.648557 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.903419 (+2254862us) service_pool.cc:225] Handling call
0320 15:27:48.961981 (+ 58562us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:49.043833 28736 tablet_bootstrap.cc:654] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.044297 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 48c1002cbdd845e2b96dcb57c450cc6d (table default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]) successfully deleted
I20260320 15:27:49.044497 30840 tablet_replica.cc:333] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.044796 30840 raft_consensus.cc:2243] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
W20260320 15:27:49.045049 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 06cb44b9cc6445cf8324cf52190d6773 because the tablet was not found. No further retry: Not found: Tablet not found: 06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:49.045106 30840 pending_rounds.cc:70] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:49.045159 30840 pending_rounds.cc:77] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387839248039936 op_type: NO_OP noop_request { }
I20260320 15:27:49.046213 30840 raft_consensus.cc:2889] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:49.046386 30840 raft_consensus.cc:2272] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.048794 30840 ts_tablet_manager.cc:1916] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.050071 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 166) took 2461 ms. Trace:
I20260320 15:27:49.050313 28900 rpcz_store.cc:276] 0320 15:27:46.588837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.841861 (+2253024us) service_pool.cc:225] Handling call
0320 15:27:48.951149 (+109288us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:48.954219 (+ 3070us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:48.975111 (+ 20892us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.049804 (+ 74693us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.049967 (+ 163us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.962112 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.962290 (+ 178us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.962301 (+ 11us) write_op.cc:435] Acquired schema lock
0320 15:27:48.962305 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:48.962928 (+ 623us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.962948 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.962957 (+ 9us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:48.963094 (+ 137us) tablet.cc:700] Row locks acquired
0320 15:27:48.963099 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:48.963148 (+ 49us) write_op.cc:270] Start()
0320 15:27:48.963206 (+ 58us) write_op.cc:276] Timestamp: P: 1774020468963116 usec, L: 0
0320 15:27:48.963211 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.963453 (+ 242us) log.cc:844] Serialized 433 byte log entry
0320 15:27:48.970104 (+ 6651us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.970224 (+ 120us) write_op.cc:301] APPLY: starting
0320 15:27:48.970245 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.970296 (+ 51us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.970327 (+ 31us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.970452 (+ 125us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.970494 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:48.970501 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:48.970587 (+ 86us) log.cc:844] Serialized 29 byte log entry
0320 15:27:48.970665 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.970687 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:48.970694 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.970769 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.970772 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":534,"apply.run_wall_time_us":534,"num_ops":2,"prepare.queue_time_us":7083,"prepare.run_cpu_time_us":1037,"prepare.run_wall_time_us":1499,"raft.queue_time_us":1509,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":307,"replication_time_us":6857}]]}
I20260320 15:27:49.052055 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 169) took 2429 ms. Trace:
I20260320 15:27:49.057637 28890 rpcz_store.cc:276] 0320 15:27:46.622382 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.857742 (+2235360us) service_pool.cc:225] Handling call
0320 15:27:48.951634 (+ 93892us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:48.958119 (+ 6485us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:48.986583 (+ 28464us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.051838 (+ 65255us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.051971 (+ 133us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.963642 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.963798 (+ 156us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.963805 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:48.963810 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:48.963898 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.963909 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.963914 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:48.963995 (+ 81us) tablet.cc:700] Row locks acquired
0320 15:27:48.963997 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:48.964033 (+ 36us) write_op.cc:270] Start()
0320 15:27:48.964090 (+ 57us) write_op.cc:276] Timestamp: P: 1774020468964002 usec, L: 0
0320 15:27:48.964093 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.964262 (+ 169us) log.cc:844] Serialized 433 byte log entry
0320 15:27:48.977499 (+ 13237us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.977798 (+ 299us) write_op.cc:301] APPLY: starting
0320 15:27:48.977821 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.977876 (+ 55us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.977909 (+ 33us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.985266 (+ 7357us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.985313 (+ 47us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:48.985322 (+ 9us) write_op.cc:312] APPLY: finished
0320 15:27:48.985449 (+ 127us) log.cc:844] Serialized 29 byte log entry
0320 15:27:48.985542 (+ 93us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.985567 (+ 25us) write_op.cc:454] Released schema lock
0320 15:27:48.985574 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:48.985666 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.985690 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":207,"apply.run_cpu_time_us":754,"apply.run_wall_time_us":7850,"mutex_wait_us":3472,"num_ops":2,"prepare.queue_time_us":4718,"prepare.run_cpu_time_us":722,"prepare.run_wall_time_us":721,"raft.queue_time_us":1093,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":4,"replication_time_us":6066}]]}
W20260320 15:27:49.053740 28901 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39290b96eadb43968da86872c4682a5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.064896 28736 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.065068 28736 ts_tablet_manager.cc:1403] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.003s
I20260320 15:27:49.065972 28736 raft_consensus.cc:359] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.066203 28736 raft_consensus.cc:385] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.066256 28736 raft_consensus.cc:740] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.066494 28736 consensus_queue.cc:260] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.068538 29643 ts_tablet_manager.cc:1929] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.068763 29643 log.cc:1199] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/28c453eaf49c4171b0f93d3b26690d83
I20260320 15:27:49.069317 29643 ts_tablet_manager.cc:1950] T 28c453eaf49c4171b0f93d3b26690d83 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.070552 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 390) took 1122 ms. Trace:
I20260320 15:27:49.070722 29643 rpcz_store.cc:276] 0320 15:27:47.947681 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.947754 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:49.035307 (+1087553us) ts_tablet_manager.cc:1151] Deleting tablet 28c453eaf49c4171b0f93d3b26690d83
0320 15:27:49.035318 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.052642 (+ 17324us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.068374 (+ 15732us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.070522 (+ 2148us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1087262}
I20260320 15:27:49.071575 29643 tablet_replica.cc:333] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.071822 29643 raft_consensus.cc:2243] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.071792 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 28c453eaf49c4171b0f93d3b26690d83 (table default.Table_78c6dd62d8834a39afa05aac3371a7c0 [id=d4b0f6f4041849bf8e605c1a1be65d79]) successfully deleted
I20260320 15:27:49.072021 29643 raft_consensus.cc:2272] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.073022 29643 ts_tablet_manager.cc:1916] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.076594 30840 ts_tablet_manager.cc:1929] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.077196 30840 log.cc:1199] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:49.077591 30840 ts_tablet_manager.cc:1950] T ae3422cf33324ab89a2b44e27cae0eb2 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.078955 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet ae3422cf33324ab89a2b44e27cae0eb2 (table default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]) successfully deleted
W20260320 15:27:49.079870 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet e779a274913f4caeaf82e3f180457830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e779a274913f4caeaf82e3f180457830 already in progress: creating tablet
I20260320 15:27:49.080122 28736 ts_tablet_manager.cc:1434] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.015s user 0.001s sys 0.002s
I20260320 15:27:49.080475 28736 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
W20260320 15:27:49.080641 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9dbd3b0e92a744dcbc132037c67303ab because the tablet was not found. No further retry: Not found: Tablet not found: 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:49.080981 29643 ts_tablet_manager.cc:1929] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.081179 29643 log.cc:1199] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/8428019c3b074b17aec493c14cc2b911
W20260320 15:27:49.081387 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d04ab44039124f36ac60601098586b4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d04ab44039124f36ac60601098586b4e already in progress: creating tablet
I20260320 15:27:49.081684 29643 ts_tablet_manager.cc:1950] T 8428019c3b074b17aec493c14cc2b911 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.082031 28736 tablet_bootstrap.cc:654] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:49.082051 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet a185f84c24ea402684aa1cc3628d4f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a185f84c24ea402684aa1cc3628d4f11 already in progress: creating tablet
W20260320 15:27:49.082692 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet aba79ddde6e842dfb67597c5d9e371cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba79ddde6e842dfb67597c5d9e371cb already in progress: creating tablet
I20260320 15:27:49.082723 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 391) took 1131 ms. Trace:
I20260320 15:27:49.082798 29643 rpcz_store.cc:276] 0320 15:27:47.951223 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.951348 (+ 125us) service_pool.cc:225] Handling call
0320 15:27:49.071505 (+1120157us) ts_tablet_manager.cc:1151] Deleting tablet 8428019c3b074b17aec493c14cc2b911
0320 15:27:49.071518 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.077449 (+ 5931us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.080861 (+ 3412us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.082701 (+ 1840us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1119845}
I20260320 15:27:49.083088 30840 tablet_replica.cc:333] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.083413 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 8428019c3b074b17aec493c14cc2b911 (table default.Table_462a9e5f58364d459620a9eec9c2b570 [id=8fd239a66a0346629cf7df7adb05e004]) successfully deleted
I20260320 15:27:49.083707 28734 raft_consensus.cc:359] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.083940 28734 raft_consensus.cc:385] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.083995 28734 raft_consensus.cc:740] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.084255 28734 consensus_queue.cc:260] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:49.084440 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6049ed1c2d84494893bfa20a9d563838 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6049ed1c2d84494893bfa20a9d563838 already in progress: creating tablet
I20260320 15:27:49.085878 28734 ts_tablet_manager.cc:1434] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.085s user 0.001s sys 0.001s
I20260320 15:27:49.086184 28734 tablet_bootstrap.cc:492] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.088271 28734 tablet_bootstrap.cc:654] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.028119 30725 consensus_queue.cc:1048] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.089416 29640 pending_rounds.cc:77] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387823937216512 op_type: NO_OP noop_request { }
I20260320 15:27:49.089586 29640 raft_consensus.cc:2889] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:49.089696 29640 raft_consensus.cc:2272] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.090324 29640 ts_tablet_manager.cc:1916] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.043115 30395 consensus_queue.cc:1048] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:49.072130 28636 tablet.cc:2379] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:49.095400 28905 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_d1c4892174e5421194d26b7a9cd25545"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.096169 28905 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1c4892174e5421194d26b7a9cd25545 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.073827 28733 tablet_bootstrap.cc:654] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.096678 30840 raft_consensus.cc:2243] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.096845 28890 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_1eeeaa1f75324b11ac9a56307a53ff99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:49.097285 30840 pending_rounds.cc:70] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:49.097339 30840 pending_rounds.cc:77] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387840973082624 op_type: NO_OP noop_request { }
W20260320 15:27:49.097422 28890 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1eeeaa1f75324b11ac9a56307a53ff99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.097427 30840 raft_consensus.cc:2889] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:49.097527 30840 raft_consensus.cc:2272] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.097705 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_d65492e9e70b4dcdb7cf91ab5d48531e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.098174 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d65492e9e70b4dcdb7cf91ab5d48531e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.098492 30840 ts_tablet_manager.cc:1916] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.100620 30456 consensus_queue.cc:1048] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.105378 30840 ts_tablet_manager.cc:1929] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.105520 30840 log.cc:1199] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:49.105896 30840 ts_tablet_manager.cc:1950] T 3ed5d220ec4b406aa5bae970f785de63 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.107098 30840 tablet_replica.cc:333] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.107336 30840 raft_consensus.cc:2243] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.107501 30840 raft_consensus.cc:2272] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.125352 29643 tablet_replica.cc:333] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.127161 30840 ts_tablet_manager.cc:1916] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.127605 29643 raft_consensus.cc:2243] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.128623 29643 raft_consensus.cc:2272] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.129848 29643 ts_tablet_manager.cc:1916] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.136372 30687 raft_consensus.cc:493] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.136682 30687 raft_consensus.cc:515] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.137995 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.138226 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:49.138610 28383 leader_election.cc:343] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.138818 28382 leader_election.cc:343] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.138954 28382 leader_election.cc:304] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.139284 30456 raft_consensus.cc:2749] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.139603 30687 leader_election.cc:290] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.144397 30767 consensus_queue.cc:1048] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.145521 28736 tablet_bootstrap.cc:492] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.145690 28736 ts_tablet_manager.cc:1403] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.065s user 0.001s sys 0.001s
W20260320 15:27:49.147418 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d04ab44039124f36ac60601098586b4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d04ab44039124f36ac60601098586b4e already in progress: creating tablet
I20260320 15:27:49.147759 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 3ed5d220ec4b406aa5bae970f785de63 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]) successfully deleted
I20260320 15:27:49.151918 30623 consensus_queue.cc:1048] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:49.153214 28521 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:49.153787 28862 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:49.153942 28862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:49.169080 30840 ts_tablet_manager.cc:1929] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.169226 30840 log.cc:1199] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:49.169552 30840 ts_tablet_manager.cc:1950] T 2e9791ac61b34fc09a8292c79f0c4032 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.170657 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 2e9791ac61b34fc09a8292c79f0c4032 (table default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]) successfully deleted
I20260320 15:27:49.170919 28736 raft_consensus.cc:359] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.171090 28736 raft_consensus.cc:385] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.171136 28736 raft_consensus.cc:740] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.171363 28736 consensus_queue.cc:260] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.171471 30868 consensus_queue.cc:1048] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.171936 28736 ts_tablet_manager.cc:1434] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.026s user 0.002s sys 0.000s
I20260320 15:27:49.179863 29640 ts_tablet_manager.cc:1929] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.179991 29640 log.cc:1199] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:49.180591 28733 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.180730 28733 ts_tablet_manager.cc:1403] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.168s user 0.007s sys 0.007s
I20260320 15:27:49.181563 28733 raft_consensus.cc:359] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.182035 28733 raft_consensus.cc:385] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.182083 28733 raft_consensus.cc:740] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.182322 28733 consensus_queue.cc:260] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.182796 28733 ts_tablet_manager.cc:1434] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:49.183063 28733 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.184542 28733 tablet_bootstrap.cc:654] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.187305 28736 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.191191 29643 ts_tablet_manager.cc:1929] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260320 15:27:49.192247 28237 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260320 15:27:49.194447 30880 negotiation.cc:337] Failed RPC negotiation. Trace:
0320 15:27:49.163361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.74.61:44739 (local address 127.27.74.3:43221)
0320 15:27:49.186219 (+ 22858us) negotiation.cc:107] Waiting for socket to connect
0320 15:27:49.186246 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0320 15:27:49.187786 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0320 15:27:49.194237 (+ 6451us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":22705,"thread_start_us":22585,"threads_started":1}
I20260320 15:27:49.195619 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:49.196689 28733 tablet_bootstrap.cc:492] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.196830 28733 ts_tablet_manager.cc:1403] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260320 15:27:49.197551 28733 raft_consensus.cc:359] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.197715 28733 raft_consensus.cc:385] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.197755 28733 raft_consensus.cc:740] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.197973 28733 consensus_queue.cc:260] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.198567 28733 ts_tablet_manager.cc:1434] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:49.198854 28733 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.200663 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:49.200979 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:49.201370 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]). Expected version 2 got 0
I20260320 15:27:49.201846 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_8ef5be02dff04c058181c86aa2de56a0 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 4 got 0
I20260320 15:27:49.202107 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:49.202387 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:49.202484 29640 ts_tablet_manager.cc:1950] T cea3bf14653f4555acfed1577a79ee4f P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.203052 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 408f5b4d67514838a17c180ae2cd71df (table default.Table_1671adae9cbe4d76a53e9544d76d5497 [id=b2921bb259f24e1682ec25e2f083ff5f]). Expected version 1 got 0
I20260320 15:27:49.203821 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 224) took 8790 ms. Trace:
I20260320 15:27:49.204008 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:49.204517 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:49.207329 28736 tablet_bootstrap.cc:654] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.208105 28733 tablet_bootstrap.cc:654] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.203949 29640 rpcz_store.cc:276] 0320 15:27:40.413219 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.413292 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:49.014533 (+8601241us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:49.014551 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.120967 (+106416us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.179732 (+ 58765us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.203776 (+ 24044us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8600730}
I20260320 15:27:49.209934 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 225) took 8795 ms. Trace:
W20260320 15:27:49.192036 28380 connection.cc:537] client connection to 127.27.74.61:44739 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260320 15:27:49.210469 29640 rpcz_store.cc:276] 0320 15:27:40.413888 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.413938 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:49.209361 (+8795423us) ts_tablet_manager.cc:1151] Deleting tablet 66a181681cd84b7fb09c3742f4f2222f
0320 15:27:49.209371 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.209763 (+ 392us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8795181}
I20260320 15:27:49.204721 28903 catalog_manager.cc:5671] T 54c75b9deab44c829f6720480e38c80b P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:49.213022 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:49.213109 29640 tablet_replica.cc:333] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.213418 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:49.209383 28734 tablet_bootstrap.cc:492] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.214088 28734 ts_tablet_manager.cc:1403] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.128s user 0.003s sys 0.003s
I20260320 15:27:49.214208 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:49.214458 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:49.214603 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:49.214874 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:49.214991 28734 raft_consensus.cc:359] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.215135 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:49.214713 29640 raft_consensus.cc:2243] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.215190 28734 raft_consensus.cc:385] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.215243 28734 raft_consensus.cc:740] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.216114 29640 raft_consensus.cc:2272] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.218140 28734 consensus_queue.cc:260] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.218675 28734 ts_tablet_manager.cc:1434] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:49.218966 28734 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.218984 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet cea3bf14653f4555acfed1577a79ee4f (table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]) successfully deleted
W20260320 15:27:49.219362 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 66a181681cd84b7fb09c3742f4f2222f because the tablet was not found. No further retry: Not found: Tablet not found: 66a181681cd84b7fb09c3742f4f2222f
I20260320 15:27:49.223121 29640 ts_tablet_manager.cc:1916] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.223887 28734 tablet_bootstrap.cc:654] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.215483 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:49.227615 28736 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.227775 28736 ts_tablet_manager.cc:1403] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.001s
I20260320 15:27:49.226323 28903 catalog_manager.cc:5671] T 4150c4c26a26405b9be16f66d15a8948 P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:49.228489 28736 raft_consensus.cc:359] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.228694 28736 raft_consensus.cc:385] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.228755 28736 raft_consensus.cc:740] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.229723 29643 log.cc:1199] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/ae3422cf33324ab89a2b44e27cae0eb2
W20260320 15:27:49.229789 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7a4b5a658cd84b04b50b54973c065889 because the tablet was not found. No further retry: Not found: Tablet not found: 7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:49.230683 30840 tablet_replica.cc:333] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.230906 30840 raft_consensus.cc:2243] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.231086 30840 raft_consensus.cc:2272] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.231215 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:49.231962 29643 ts_tablet_manager.cc:1950] T ae3422cf33324ab89a2b44e27cae0eb2 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.235313 28733 tablet_bootstrap.cc:492] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.235450 30840 ts_tablet_manager.cc:1916] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.236347 28733 ts_tablet_manager.cc:1403] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.000s
I20260320 15:27:49.237462 28736 consensus_queue.cc:260] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.239094 29643 tablet_replica.cc:333] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.239090 28733 raft_consensus.cc:359] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.239388 28733 raft_consensus.cc:385] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.239491 28733 raft_consensus.cc:740] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.239799 28733 consensus_queue.cc:260] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.240427 28733 ts_tablet_manager.cc:1434] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:49.233704 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet ae3422cf33324ab89a2b44e27cae0eb2 (table default.Table_fd26120a419a45efa62bf54b58fd74a1 [id=31ae50a5b6a04d34b4a856d16a04f48b]) successfully deleted
I20260320 15:27:49.240676 29643 raft_consensus.cc:2243] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.240837 29643 raft_consensus.cc:2272] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:49.240888 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e779a274913f4caeaf82e3f180457830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e779a274913f4caeaf82e3f180457830 already in progress: creating tablet
I20260320 15:27:49.241351 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:49.241430 29643 ts_tablet_manager.cc:1916] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.242416 30623 consensus_queue.cc:1048] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.243060 28736 ts_tablet_manager.cc:1434] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260320 15:27:49.244211 28733 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.244727 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:49.246685 28733 tablet_bootstrap.cc:654] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.247601 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]). Expected version 1 got 0
I20260320 15:27:49.248181 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:49.248598 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:49.248929 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:49.249287 28736 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.250238 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:49.251092 28736 tablet_bootstrap.cc:654] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.251362 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:49.251664 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:49.248971 28734 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.255318 28734 ts_tablet_manager.cc:1403] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.036s user 0.005s sys 0.000s
I20260320 15:27:49.256175 28734 raft_consensus.cc:359] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.256353 28734 raft_consensus.cc:385] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.256408 28734 raft_consensus.cc:740] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.256997 28903 catalog_manager.cc:5671] T f6240d441c624b238f7c13eb3c5fa9fc P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:49.261431 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
W20260320 15:27:49.267242 28524 consensus_peers.cc:597] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 90cd18e2ccbf48c8b804329bd3abf847. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.274493 28582 raft_consensus.cc:3060] T 7dfbc7338eda489987315e681ee44146 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.275936 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:49.276573 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:49.276993 28903 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:49.277420 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet d5095b42b69d40539118c1f63a95f795
I20260320 15:27:49.278069 28889 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 167) took 2685 ms. Trace:
I20260320 15:27:49.278435 30852 consensus_queue.cc:1048] T 7dfbc7338eda489987315e681ee44146 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.278187 28889 rpcz_store.cc:276] 0320 15:27:46.592398 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.850683 (+2258285us) service_pool.cc:225] Handling call
0320 15:27:48.950751 (+100068us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:48.950953 (+ 202us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:48.951041 (+ 88us) catalog_manager.cc:2824] Locking tablets
0320 15:27:48.959173 (+ 8132us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:49.016473 (+ 57300us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:49.051629 (+ 35156us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:49.277365 (+225736us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:49.278001 (+ 636us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.964399 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.964489 (+ 90us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.964494 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:48.964496 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:48.964600 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.964609 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.964613 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:48.964704 (+ 91us) tablet.cc:700] Row locks acquired
0320 15:27:48.964707 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:48.964739 (+ 32us) write_op.cc:270] Start()
0320 15:27:48.964765 (+ 26us) write_op.cc:276] Timestamp: P: 1774020468964712 usec, L: 0
0320 15:27:48.964768 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.964929 (+ 161us) log.cc:844] Serialized 759 byte log entry
0320 15:27:48.977668 (+ 12739us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.986837 (+ 9169us) write_op.cc:301] APPLY: starting
0320 15:27:48.986860 (+ 23us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.986914 (+ 54us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.015908 (+ 28994us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.016010 (+ 102us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.016052 (+ 42us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:49.016062 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:49.016162 (+ 100us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.016257 (+ 95us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.016285 (+ 28us) write_op.cc:454] Released schema lock
0320 15:27:49.016293 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.098374 (+ 82081us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.098383 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9113,"apply.run_cpu_time_us":621,"apply.run_wall_time_us":29531,"mutex_wait_us":2427,"num_ops":2,"prepare.queue_time_us":4385,"prepare.run_cpu_time_us":10904,"prepare.run_wall_time_us":133928,"raft.queue_time_us":457,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":12870,"thread_start_us":201128,"threads_started":2}]]}
I20260320 15:27:49.279062 28889 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_ef3eff1b17b34682941f0ec4b03358ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:49.284183 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet d5095b42b69d40539118c1f63a95f795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.284497 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet d5095b42b69d40539118c1f63a95f795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.284526 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet d5095b42b69d40539118c1f63a95f795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:48 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:49.284929 28889 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef3eff1b17b34682941f0ec4b03358ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.287248 29640 ts_tablet_manager.cc:1929] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.287890 28734 consensus_queue.cc:260] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.293111 29640 log.cc:1199] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/3ed5d220ec4b406aa5bae970f785de63
I20260320 15:27:49.293701 28734 ts_tablet_manager.cc:1434] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.038s user 0.000s sys 0.002s
I20260320 15:27:49.295460 28734 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.299856 28734 tablet_bootstrap.cc:654] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.299713 28733 tablet_bootstrap.cc:492] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.300817 28733 ts_tablet_manager.cc:1403] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.003s
I20260320 15:27:49.301541 28733 raft_consensus.cc:359] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.301795 28733 raft_consensus.cc:385] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.301899 28733 raft_consensus.cc:740] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.302181 28733 consensus_queue.cc:260] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.303892 28733 ts_tablet_manager.cc:1434] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260320 15:27:49.305205 28733 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.306811 28733 tablet_bootstrap.cc:654] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.311174 30840 ts_tablet_manager.cc:1929] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.311362 30840 log.cc:1199] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:49.311756 30840 ts_tablet_manager.cc:1950] T e2243e6811744b7cba99a487f8e8bb83 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.306670 29640 ts_tablet_manager.cc:1950] T 3ed5d220ec4b406aa5bae970f785de63 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.312456 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet e2243e6811744b7cba99a487f8e8bb83 (table default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]) successfully deleted
I20260320 15:27:49.312841 30840 tablet_replica.cc:333] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.313539 30840 raft_consensus.cc:2243] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.314354 30840 raft_consensus.cc:2272] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.315201 30840 ts_tablet_manager.cc:1916] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.315896 29643 ts_tablet_manager.cc:1929] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.316043 29643 log.cc:1199] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/48c1002cbdd845e2b96dcb57c450cc6d
I20260320 15:27:49.314643 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 229) took 8793 ms. Trace:
I20260320 15:27:49.316349 29643 ts_tablet_manager.cc:1950] T 48c1002cbdd845e2b96dcb57c450cc6d P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.317286 28734 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.318674 28734 ts_tablet_manager.cc:1403] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.002s
I20260320 15:27:49.319648 28734 raft_consensus.cc:359] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.319907 28734 raft_consensus.cc:385] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.320004 28734 raft_consensus.cc:740] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.320396 28734 consensus_queue.cc:260] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.316385 29640 rpcz_store.cc:276] 0320 15:27:40.520804 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.520867 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:49.212510 (+8691643us) ts_tablet_manager.cc:1151] Deleting tablet 3ed5d220ec4b406aa5bae970f785de63
0320 15:27:49.212886 (+ 376us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.264777 (+ 51891us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.287081 (+ 22304us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.314320 (+ 27239us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8688482}
I20260320 15:27:49.321722 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 3ed5d220ec4b406aa5bae970f785de63 (table default.Table_283c1ab1e64449abaa1696a2d2bc7911 [id=a26130d99d8a4f98bb99c3bff5c7017c]) successfully deleted
I20260320 15:27:49.322697 30868 consensus_queue.cc:1048] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.323905 28734 ts_tablet_manager.cc:1434] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:49.325197 28734 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.325811 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 236) took 8463 ms. Trace:
I20260320 15:27:49.328184 29640 rpcz_store.cc:276] 0320 15:27:40.861799 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:40.861873 (+ 74us) service_pool.cc:225] Handling call
0320 15:27:49.323316 (+8461443us) ts_tablet_manager.cc:1151] Deleting tablet e779a274913f4caeaf82e3f180457830
0320 15:27:49.323654 (+ 338us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.325400 (+ 1746us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8460883}
I20260320 15:27:49.332912 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 48c1002cbdd845e2b96dcb57c450cc6d (table default.Table_5a50b92df46f43e8ac3618740428b8e4 [id=90ad4efaded04091bfe6ba35eca59936]) successfully deleted
W20260320 15:27:49.333210 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet e779a274913f4caeaf82e3f180457830 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e779a274913f4caeaf82e3f180457830 already in progress: creating tablet
W20260320 15:27:49.334725 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 90cd18e2ccbf48c8b804329bd3abf847 because the tablet was not found. No further retry: Not found: Tablet not found: 90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:49.335357 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dbd3b0e92a744dcbc132037c67303ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:39 UTC) from {username='slave'} at 127.0.0.1:36478
W20260320 15:27:49.335436 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec already in progress: creating tablet
I20260320 15:27:49.335748 29643 tablet_replica.cc:333] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.337090 29640 tablet_replica.cc:333] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.340483 29643 raft_consensus.cc:2243] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.345396 29640 raft_consensus.cc:2243] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.350869 28901 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 172) took 2698 ms. Trace:
I20260320 15:27:49.352129 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 170) took 2719 ms. Trace:
I20260320 15:27:49.340891 28734 tablet_bootstrap.cc:654] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.352691 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 4b527973ede040e286f7b8f2e7536699 (DEFAULT_TABLE table=default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.353180 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 4b527973ede040e286f7b8f2e7536699 (DEFAULT_TABLE table=default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.352350 28906 rpcz_store.cc:276] 0320 15:27:46.632773 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.868967 (+2236194us) service_pool.cc:225] Handling call
0320 15:27:48.951728 (+ 82761us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:48.961139 (+ 9411us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.123942 (+162803us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.351985 (+228043us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.352078 (+ 93us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.098495 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.098602 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.098607 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:49.098609 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.098695 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.098709 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.098714 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.098803 (+ 89us) tablet.cc:700] Row locks acquired
0320 15:27:49.098805 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.098849 (+ 44us) write_op.cc:270] Start()
0320 15:27:49.098890 (+ 41us) write_op.cc:276] Timestamp: P: 1774020469098807 usec, L: 0
0320 15:27:49.098896 (+ 6us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.099093 (+ 197us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.117275 (+ 18182us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.117461 (+ 186us) write_op.cc:301] APPLY: starting
0320 15:27:49.117482 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.117534 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.117575 (+ 41us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.117701 (+ 126us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.117746 (+ 45us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.117754 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:49.117864 (+ 110us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.117948 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.117972 (+ 24us) write_op.cc:454] Released schema lock
0320 15:27:49.117979 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.118064 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.118068 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":594,"apply.run_wall_time_us":593,"mutex_wait_us":698,"num_ops":2,"prepare.queue_time_us":136701,"prepare.run_cpu_time_us":683,"prepare.run_wall_time_us":683,"replication_time_us":18337}]]}
I20260320 15:27:49.353513 29643 raft_consensus.cc:2272] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.354079 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 7a4b5a658cd84b04b50b54973c065889 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:49.354041 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 4b527973ede040e286f7b8f2e7536699 (DEFAULT_TABLE table=default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.354470 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b527973ede040e286f7b8f2e7536699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.354741 28906 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 177) took 2647 ms. Trace:
I20260320 15:27:49.354921 28906 rpcz_store.cc:276] 0320 15:27:46.707185 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.353968 (+2646783us) service_pool.cc:225] Handling call
0320 15:27:49.354668 (+ 700us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260320 15:27:49.355507 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_d24ebbeb63cd41139ef872dfd11b5545"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.356364 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d24ebbeb63cd41139ef872dfd11b5545 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.354475 29643 ts_tablet_manager.cc:1916] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.358336 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b527973ede040e286f7b8f2e7536699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.359491 29640 raft_consensus.cc:2272] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.361958 29640 ts_tablet_manager.cc:1916] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.369590 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet cea3bf14653f4555acfed1577a79ee4f with delete_type TABLET_DATA_DELETED (replaced by tablet d04ab44039124f36ac60601098586b4e) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.351084 28901 rpcz_store.cc:276] 0320 15:27:46.652357 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.950568 (+2298211us) service_pool.cc:225] Handling call
0320 15:27:49.060153 (+109585us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.060774 (+ 621us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.124126 (+ 63352us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.350690 (+226564us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.350818 (+ 128us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.099210 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.099313 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.099319 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:49.099321 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.099390 (+ 69us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.099399 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.099403 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.099463 (+ 60us) tablet.cc:700] Row locks acquired
0320 15:27:49.099465 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.099493 (+ 28us) write_op.cc:270] Start()
0320 15:27:49.099521 (+ 28us) write_op.cc:276] Timestamp: P: 1774020469099454 usec, L: 0
0320 15:27:49.099524 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.102719 (+ 3195us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x6160000a33b8. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe612ea4775 00007fe612f8951b 00007fe612f64518 00007fe612f63dc5 00007fe61434a795 00007fe6143492b9 00007fe6143516e0 00007fe6143514ed 0000000000a8b58c 00007fe6070d4f5f
0320 15:27:49.103121 (+ 402us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.117353 (+ 14232us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.118142 (+ 789us) write_op.cc:301] APPLY: starting
0320 15:27:49.118153 (+ 11us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.118191 (+ 38us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.118194 (+ 3us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.118309 (+ 115us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.118338 (+ 29us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.118345 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.118410 (+ 65us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.118470 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.118486 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:49.118492 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.118551 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.118554 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":755,"apply.run_cpu_time_us":404,"apply.run_wall_time_us":403,"num_ops":2,"prepare.queue_time_us":37853,"prepare.run_cpu_time_us":1212,"prepare.run_wall_time_us":4014,"replication_time_us":17785,"spinlock_wait_cycles":6048640}]]}
I20260320 15:27:49.370600 28901 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_cf6e9715cac5401dbfda04f61b2d3ae3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.371428 28901 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf6e9715cac5401dbfda04f61b2d3ae3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.357892 28905 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 173) took 2705 ms. Trace:
I20260320 15:27:49.372226 28905 rpcz_store.cc:276] 0320 15:27:46.652584 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.043501 (+2390917us) service_pool.cc:225] Handling call
0320 15:27:49.095307 (+ 51806us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.338634 (+243327us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.339194 (+ 560us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.356679 (+ 17485us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.357699 (+ 1020us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.357800 (+ 101us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.339981 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.340120 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.340128 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:49.340131 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:49.340234 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.340252 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.340262 (+ 10us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.340359 (+ 97us) tablet.cc:700] Row locks acquired
0320 15:27:49.340363 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:49.340405 (+ 42us) write_op.cc:270] Start()
0320 15:27:49.340444 (+ 39us) write_op.cc:276] Timestamp: P: 1774020469340341 usec, L: 0
0320 15:27:49.340448 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.340669 (+ 221us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.347230 (+ 6561us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.347400 (+ 170us) write_op.cc:301] APPLY: starting
0320 15:27:49.347421 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.347474 (+ 53us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.347503 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.347678 (+ 175us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.347722 (+ 44us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.347734 (+ 12us) write_op.cc:312] APPLY: finished
0320 15:27:49.347834 (+ 100us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.347913 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.347944 (+ 31us) write_op.cc:454] Released schema lock
0320 15:27:49.347954 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.348038 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.348042 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":117750656,"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":626,"apply.run_wall_time_us":625,"num_ops":2,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":814,"prepare.run_wall_time_us":814,"raft.queue_time_us":73,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":291,"replication_time_us":6739}]]}
I20260320 15:27:49.373276 28905 master_service.cc:954] ReplaceTablet: received request to replace tablet 9d7f7b99899c4001adcbdce58910e717 from {username='slave'} at 127.0.0.1:33394
W20260320 15:27:49.375782 28862 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.61:44739, user_credentials={real_user=slave}} blocked reactor thread for 221844us
W20260320 15:27:49.377612 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55bd68c6e4364a038a815d1fc05ad25e consensus state has no leader
W20260320 15:27:49.380734 28904 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.27.74.3:43105 (request call id 104) took 1553 ms (client timeout 1000 ms). Trace:
W20260320 15:27:49.381012 28904 rpcz_store.cc:269] 0320 15:27:47.827022 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:47.965841 (+138819us) service_pool.cc:225] Handling call
0320 15:27:48.881066 (+915225us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.881073 (+ 7us) write_op.cc:454] Released schema lock
0320 15:27:49.380662 (+499589us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:48.871627 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:48.871773 (+ 146us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:48.871781 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:48.871784 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:48.872143 (+ 359us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:48.872162 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:48.872168 (+ 6us) tablet.cc:684] Acquiring locks for 10 operations
0320 15:27:48.872386 (+ 218us) tablet.cc:700] Row locks acquired
0320 15:27:48.872390 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:48.872438 (+ 48us) write_op.cc:270] Start()
0320 15:27:48.872480 (+ 42us) write_op.cc:276] Timestamp: P: 1774020468872415 usec, L: 0
0320 15:27:48.872484 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:48.872744 (+ 260us) log.cc:844] Serialized 3894 byte log entry
0320 15:27:48.877264 (+ 4520us) op_driver.cc:464] REPLICATION: finished
0320 15:27:48.877465 (+ 201us) write_op.cc:301] APPLY: starting
0320 15:27:48.877489 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:48.877562 (+ 73us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:48.877601 (+ 39us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:48.877903 (+ 302us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:48.877952 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0320 15:27:48.877962 (+ 10us) write_op.cc:312] APPLY: finished
0320 15:27:48.878077 (+ 115us) log.cc:844] Serialized 77 byte log entry
0320 15:27:48.878160 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:48.878194 (+ 34us) write_op.cc:454] Released schema lock
0320 15:27:48.878202 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":9207,"reported_tablets":183,"child_traces":[["op",{"apply.queue_time_us":113,"apply.run_cpu_time_us":800,"apply.run_wall_time_us":799,"num_ops":10,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":1297,"prepare.run_wall_time_us":1324,"raft.queue_time_us":82,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":296,"replication_time_us":4637}]]}
I20260320 15:27:49.381649 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b527973ede040e286f7b8f2e7536699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.396183 30840 ts_tablet_manager.cc:1929] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.398525 28906 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 178) took 2580 ms. Trace:
I20260320 15:27:49.398794 28906 rpcz_store.cc:276] 0320 15:27:46.818411 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.355257 (+2536846us) service_pool.cc:225] Handling call
0320 15:27:49.369634 (+ 14377us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.385585 (+ 15951us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:49.385771 (+ 186us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.398144 (+ 12373us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.398302 (+ 158us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.398428 (+ 126us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.387131 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.387299 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.387308 (+ 9us) write_op.cc:435] Acquired schema lock
0320 15:27:49.387312 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:49.387423 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.387439 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.387447 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.387670 (+ 223us) tablet.cc:700] Row locks acquired
0320 15:27:49.387673 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.387724 (+ 51us) write_op.cc:270] Start()
0320 15:27:49.387771 (+ 47us) write_op.cc:276] Timestamp: P: 1774020469387693 usec, L: 0
0320 15:27:49.387776 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.387980 (+ 204us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.393137 (+ 5157us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.393967 (+ 830us) write_op.cc:301] APPLY: starting
0320 15:27:49.393980 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.394021 (+ 41us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.394049 (+ 28us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.394157 (+ 108us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.394193 (+ 36us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.394214 (+ 21us) write_op.cc:312] APPLY: finished
0320 15:27:49.394277 (+ 63us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.394340 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.394360 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:49.394367 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.394445 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.394451 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":33895680,"child_traces":[["op",{"apply.queue_time_us":791,"apply.run_cpu_time_us":465,"apply.run_wall_time_us":465,"num_ops":2,"prepare.queue_time_us":954,"prepare.run_cpu_time_us":911,"prepare.run_wall_time_us":2117,"raft.queue_time_us":611,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":5339}]]}
I20260320 15:27:49.400167 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 1ccfd2d5e82448b99f1022954b98c6cc from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:49.403877 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 9d7f7b99899c4001adcbdce58910e717
I20260320 15:27:49.405529 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d7f7b99899c4001adcbdce58910e717 with delete_type TABLET_DATA_DELETED (replaced by tablet 992975b892e744bbad9bedddaf63c1a2) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.405678 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d7f7b99899c4001adcbdce58910e717 with delete_type TABLET_DATA_DELETED (replaced by tablet 992975b892e744bbad9bedddaf63c1a2) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.405972 28905 catalog_manager.cc:6741] ReplaceTablet: tablet 9d7f7b99899c4001adcbdce58910e717 deleted and replaced by tablet 992975b892e744bbad9bedddaf63c1a2
I20260320 15:27:49.406209 28905 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 180) took 2245 ms. Trace:
I20260320 15:27:49.407215 28734 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.407779 28734 ts_tablet_manager.cc:1403] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.083s user 0.001s sys 0.003s
I20260320 15:27:49.408121 28905 rpcz_store.cc:276] 0320 15:27:47.161180 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.373146 (+2211966us) service_pool.cc:225] Handling call
0320 15:27:49.403589 (+ 30443us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.403595 (+ 6us) write_op.cc:454] Released schema lock
0320 15:27:49.406118 (+ 2523us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.386155 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.386339 (+ 184us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.386349 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:49.386352 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:49.386496 (+ 144us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.386517 (+ 21us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.386525 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.386628 (+ 103us) tablet.cc:700] Row locks acquired
0320 15:27:49.386630 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.386673 (+ 43us) write_op.cc:270] Start()
0320 15:27:49.386722 (+ 49us) write_op.cc:276] Timestamp: P: 1774020469386644 usec, L: 0
0320 15:27:49.386727 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.386939 (+ 212us) log.cc:844] Serialized 622 byte log entry
0320 15:27:49.393048 (+ 6109us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.393255 (+ 207us) write_op.cc:301] APPLY: starting
0320 15:27:49.393280 (+ 25us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.393346 (+ 66us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.393403 (+ 57us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.393535 (+ 132us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.393615 (+ 80us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:49.393626 (+ 11us) write_op.cc:312] APPLY: finished
0320 15:27:49.393727 (+ 101us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.393821 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.393848 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:49.393857 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":145,"apply.run_cpu_time_us":666,"apply.run_wall_time_us":665,"num_ops":2,"prepare.queue_time_us":11694,"prepare.run_cpu_time_us":927,"prepare.run_wall_time_us":927,"raft.queue_time_us":1074,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":493,"replication_time_us":6277}]]}
I20260320 15:27:49.408660 28734 raft_consensus.cc:359] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.408843 28734 raft_consensus.cc:385] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.408908 28734 raft_consensus.cc:740] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.408982 29640 ts_tablet_manager.cc:1929] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:49.409116 29640 log.cc:1199] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/35347116811e4aee8142a503dd21843a
I20260320 15:27:49.409514 29640 ts_tablet_manager.cc:1950] T 35347116811e4aee8142a503dd21843a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.409277 28734 consensus_queue.cc:260] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.412004 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 1ccfd2d5e82448b99f1022954b98c6cc
I20260320 15:27:49.412637 28734 ts_tablet_manager.cc:1434] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:49.412799 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc with delete_type TABLET_DATA_DELETED (replaced by tablet 4321cb0619f24a0da475390e9a69491f) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.412838 28890 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 175) took 2725 ms. Trace:
I20260320 15:27:49.413403 28906 catalog_manager.cc:6741] ReplaceTablet: tablet 1ccfd2d5e82448b99f1022954b98c6cc deleted and replaced by tablet 4321cb0619f24a0da475390e9a69491f
I20260320 15:27:49.413535 28906 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:33394 (request call id 181) took 2121 ms. Trace:
I20260320 15:27:49.412972 28890 rpcz_store.cc:276] 0320 15:27:46.687149 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.059379 (+2372230us) service_pool.cc:225] Handling call
0320 15:27:49.096752 (+ 37373us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.357312 (+260560us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.382949 (+ 25637us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:49.383436 (+ 487us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.412509 (+ 29073us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.412658 (+ 149us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.412769 (+ 111us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.402559 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.402661 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.402667 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:49.402670 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:49.402745 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.402756 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.402761 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.402850 (+ 89us) tablet.cc:700] Row locks acquired
0320 15:27:49.402853 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.402883 (+ 30us) write_op.cc:270] Start()
0320 15:27:49.402912 (+ 29us) write_op.cc:276] Timestamp: P: 1774020469402857 usec, L: 0
0320 15:27:49.402916 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.403097 (+ 181us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.411049 (+ 7952us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.411891 (+ 842us) write_op.cc:301] APPLY: starting
0320 15:27:49.411904 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.411963 (+ 59us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.412001 (+ 38us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.412109 (+ 108us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.412142 (+ 33us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.412149 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.412215 (+ 66us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.412276 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.412291 (+ 15us) write_op.cc:454] Released schema lock
0320 15:27:49.412296 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.412394 (+ 98us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.412398 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":143104640,"child_traces":[["op",{"apply.queue_time_us":805,"apply.run_cpu_time_us":493,"apply.run_wall_time_us":468,"mutex_wait_us":17536,"num_ops":2,"prepare.queue_time_us":18627,"prepare.run_cpu_time_us":638,"prepare.run_wall_time_us":1541,"raft.queue_time_us":368,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":8113,"spinlock_wait_cycles":37760}]]}
I20260320 15:27:49.413637 28906 rpcz_store.cc:276] 0320 15:27:47.292556 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.400045 (+2107489us) service_pool.cc:225] Handling call
0320 15:27:49.413494 (+ 13449us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.401620 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.401750 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.401755 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:49.401757 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.402030 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.402047 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.402052 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.402131 (+ 79us) tablet.cc:700] Row locks acquired
0320 15:27:49.402133 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.402174 (+ 41us) write_op.cc:270] Start()
0320 15:27:49.402209 (+ 35us) write_op.cc:276] Timestamp: P: 1774020469402145 usec, L: 0
0320 15:27:49.402214 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.402403 (+ 189us) log.cc:844] Serialized 656 byte log entry
0320 15:27:49.410963 (+ 8560us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.411154 (+ 191us) write_op.cc:301] APPLY: starting
0320 15:27:49.411174 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.411226 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.411256 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.411383 (+ 127us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.411451 (+ 68us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:49.411458 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.411589 (+ 131us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.411668 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.411695 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:49.411704 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.411792 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.411796 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":133,"apply.run_cpu_time_us":630,"apply.run_wall_time_us":629,"num_ops":2,"prepare.queue_time_us":206,"prepare.run_cpu_time_us":774,"prepare.run_wall_time_us":910,"raft.queue_time_us":710,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":283,"replication_time_us":8704}]]}
I20260320 15:27:49.413978 28890 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8d69a7f0253d4f47a24fcabe3b0dff20" } modify_external_catalogs: true
I20260320 15:27:49.414165 28890 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_8d69a7f0253d4f47a24fcabe3b0dff20" } modify_external_catalogs: true
I20260320 15:27:49.414734 28734 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.416567 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc with delete_type TABLET_DATA_DELETED (replaced by tablet 4321cb0619f24a0da475390e9a69491f) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.424109 28900 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 174) took 2739 ms. Trace:
I20260320 15:27:49.424258 28900 rpcz_store.cc:276] 0320 15:27:46.684445 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.056971 (+2372526us) service_pool.cc:225] Handling call
0320 15:27:49.097625 (+ 40654us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf6c03 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.356898 (+259273us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.384076 (+ 27178us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:49.384270 (+ 194us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.423767 (+ 39497us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.423912 (+ 145us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.424032 (+ 120us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.404146 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.404285 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.404291 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:49.404293 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.404375 (+ 82us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.404388 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.404395 (+ 7us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.404625 (+ 230us) tablet.cc:700] Row locks acquired
0320 15:27:49.404629 (+ 4us) write_op.cc:260] PREPARE: finished
0320 15:27:49.404680 (+ 51us) write_op.cc:270] Start()
0320 15:27:49.404722 (+ 42us) write_op.cc:276] Timestamp: P: 1774020469404647 usec, L: 0
0320 15:27:49.404725 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.404901 (+ 176us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.422902 (+ 18001us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.423096 (+ 194us) write_op.cc:301] APPLY: starting
0320 15:27:49.423115 (+ 19us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.423164 (+ 49us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.423196 (+ 32us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.423322 (+ 126us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.423357 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.423364 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.423462 (+ 98us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.423569 (+ 107us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.423589 (+ 20us) write_op.cc:454] Released schema lock
0320 15:27:49.423596 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.423672 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.423676 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":141243648,"child_traces":[["op",{"apply.queue_time_us":128,"apply.run_cpu_time_us":577,"apply.run_wall_time_us":578,"mutex_wait_us":17338,"num_ops":2,"prepare.queue_time_us":19439,"prepare.run_cpu_time_us":827,"prepare.run_wall_time_us":827,"replication_time_us":18130,"spinlock_wait_cycles":14720}]]}
I20260320 15:27:49.404757 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d7f7b99899c4001adcbdce58910e717 with delete_type TABLET_DATA_DELETED (replaced by tablet 992975b892e744bbad9bedddaf63c1a2) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.406198 30466 raft_consensus.cc:493] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.425042 30466 raft_consensus.cc:515] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.425549 28889 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 176) took 2731 ms. Trace:
I20260320 15:27:49.425925 30466 leader_election.cc:290] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.409757 28294 raft_consensus.cc:3060] T 547d82cee19b4959b3a779c8b8088e83 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.425657 28889 rpcz_store.cc:276] 0320 15:27:46.693750 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.278847 (+2585097us) service_pool.cc:225] Handling call
0320 15:27:49.351572 (+ 72725us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.384882 (+ 33310us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:49.385065 (+ 183us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.425172 (+ 40107us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.425289 (+ 117us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.425398 (+ 109us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.405013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.405116 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.405121 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:49.405123 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.405213 (+ 90us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.405225 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.405229 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.405312 (+ 83us) tablet.cc:700] Row locks acquired
0320 15:27:49.405314 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.405348 (+ 34us) write_op.cc:270] Start()
0320 15:27:49.405377 (+ 29us) write_op.cc:276] Timestamp: P: 1774020469405320 usec, L: 0
0320 15:27:49.405381 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.405553 (+ 172us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.422995 (+ 17442us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.423315 (+ 320us) write_op.cc:301] APPLY: starting
0320 15:27:49.423328 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.423359 (+ 31us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.423388 (+ 29us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.423456 (+ 68us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.423483 (+ 27us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.423491 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:49.423559 (+ 68us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.423591 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.423600 (+ 9us) write_op.cc:454] Released schema lock
0320 15:27:49.423603 (+ 3us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.423672 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.423676 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":75638656,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":348,"apply.run_wall_time_us":348,"mutex_wait_us":16108,"num_ops":2,"prepare.queue_time_us":19546,"prepare.run_cpu_time_us":606,"prepare.run_wall_time_us":604,"replication_time_us":17592}]]}
I20260320 15:27:49.426654 30395 raft_consensus.cc:493] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.426793 30395 raft_consensus.cc:515] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.427567 30395 leader_election.cc:290] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.413887 28281 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ccfd2d5e82448b99f1022954b98c6cc with delete_type TABLET_DATA_DELETED (replaced by tablet 4321cb0619f24a0da475390e9a69491f) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.417428 28736 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.427889 28736 ts_tablet_manager.cc:1403] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.179s user 0.006s sys 0.005s
I20260320 15:27:49.428337 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.428328 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.428532 28447 raft_consensus.cc:2468] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:49.428537 28297 raft_consensus.cc:2468] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:49.429020 28524 leader_election.cc:304] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.428938 28736 raft_consensus.cc:359] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.429137 28736 raft_consensus.cc:385] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.429198 28736 raft_consensus.cc:740] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.429459 28736 consensus_queue.cc:260] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.429587 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.430145 28736 ts_tablet_manager.cc:1434] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:49.430094 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.430302 28447 raft_consensus.cc:2468] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:49.430554 28736 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.431972 28240 leader_election.cc:304] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.432396 30466 raft_consensus.cc:2804] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.432482 30466 raft_consensus.cc:493] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.432571 30466 raft_consensus.cc:3060] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.429587 30395 raft_consensus.cc:2804] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.432808 30395 raft_consensus.cc:493] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.432900 30395 raft_consensus.cc:3060] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:49.438856 28352 tablet.cc:2379] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260320 15:27:49.439808 30655 consensus_queue.cc:1048] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.439996 30466 raft_consensus.cc:515] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.440928 30395 raft_consensus.cc:515] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.441116 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.441318 28575 raft_consensus.cc:2393] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:49.441332 28447 raft_consensus.cc:3060] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.443485 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.443619 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:49.443823 28297 raft_consensus.cc:2393] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:49.443867 30466 leader_election.cc:290] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.444432 28524 leader_election.cc:304] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.444748 30655 raft_consensus.cc:2749] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.445075 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233f6a8e9369411b8e0c35a167c55ff6" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:49.445108 28447 raft_consensus.cc:2468] T 233f6a8e9369411b8e0c35a167c55ff6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:49.445335 28575 raft_consensus.cc:2393] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:49.445739 28240 leader_election.cc:304] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.445816 28901 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:33394 (request call id 179) took 2625 ms. Trace:
I20260320 15:27:49.445990 30395 leader_election.cc:290] T 233f6a8e9369411b8e0c35a167c55ff6 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.445945 28901 rpcz_store.cc:276] 0320 15:27:46.820566 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.370364 (+2549798us) service_pool.cc:225] Handling call
0320 15:27:49.371578 (+ 1214us) catalog_manager.cc:2101] Acquired catalog manager lock
0320 15:27:49.381626 (+ 10048us) spinlock_profiling.cc:243] Waited 9.35 ms on lock 0x606000048f58. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 000000000098694a 00007fe606f9e8e5 00007fe606f9e610 00007fe61abf656c 00007fe61abef5c5 00007fe61abe819d 00007fe61abf316f 00007fe61adf6d5a 00007fe6118e57c6 00007fe6118e54f5 00007fe60e04de3e
0320 15:27:49.382160 (+ 534us) catalog_manager.cc:2147] Created new table and tablet info
0320 15:27:49.445458 (+ 63298us) catalog_manager.cc:2207] Wrote table and tablets to system table
0320 15:27:49.445628 (+ 170us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0320 15:27:49.445754 (+ 126us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.409910 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.410073 (+ 163us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.410080 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:49.410082 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.410181 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.410193 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.410197 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.410278 (+ 81us) tablet.cc:700] Row locks acquired
0320 15:27:49.410281 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.410334 (+ 53us) write_op.cc:270] Start()
0320 15:27:49.410372 (+ 38us) write_op.cc:276] Timestamp: P: 1774020469410306 usec, L: 0
0320 15:27:49.410375 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.410560 (+ 185us) log.cc:844] Serialized 433 byte log entry
0320 15:27:49.423083 (+ 12523us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.423756 (+ 673us) write_op.cc:301] APPLY: starting
0320 15:27:49.423766 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.423799 (+ 33us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.423823 (+ 24us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.423942 (+ 119us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.423971 (+ 29us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0320 15:27:49.423977 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:49.424037 (+ 60us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.424110 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.424126 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:49.424134 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":21497216,"child_traces":[["op",{"apply.queue_time_us":631,"apply.run_cpu_time_us":448,"apply.run_wall_time_us":448,"mutex_wait_us":27037,"num_ops":2,"prepare.queue_time_us":27118,"prepare.run_cpu_time_us":766,"prepare.run_wall_time_us":824,"raft.queue_time_us":35,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":12690,"threads_started":1}]]}
I20260320 15:27:49.446863 30852 raft_consensus.cc:2804] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.446982 30852 raft_consensus.cc:697] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:49.447221 30852 consensus_queue.cc:237] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.447439 28736 tablet_bootstrap.cc:654] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.448360 30773 raft_consensus.cc:493] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.448482 30773 raft_consensus.cc:515] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.448997 30773 leader_election.cc:290] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.449537 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 240) took 8012 ms. Trace:
I20260320 15:27:49.449668 29640 rpcz_store.cc:276] 0320 15:27:41.436639 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.436702 (+ 63us) service_pool.cc:225] Handling call
0320 15:27:49.336071 (+7899369us) ts_tablet_manager.cc:1151] Deleting tablet 35347116811e4aee8142a503dd21843a
0320 15:27:49.336415 (+ 344us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.375376 (+ 38961us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.408862 (+ 33486us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.449509 (+ 40647us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2304,"tablet-delete.queue_time_us":7898832}
I20260320 15:27:49.450244 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 241) took 7945 ms. Trace:
I20260320 15:27:49.450336 29640 rpcz_store.cc:276] 0320 15:27:41.504682 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.504743 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:49.450050 (+7945307us) ts_tablet_manager.cc:1151] Deleting tablet 87cb35cefee841d0881b08c456df5a97
0320 15:27:49.450083 (+ 33us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.450214 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7945113}
I20260320 15:27:49.451049 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 242) took 7915 ms. Trace:
I20260320 15:27:49.451680 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 35347116811e4aee8142a503dd21843a (table default.Table_37e1fe279d384188986393c967dcb9de [id=695e9628a1d947dd981b9f954f3f3072]) successfully deleted
W20260320 15:27:49.452028 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 87cb35cefee841d0881b08c456df5a97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87cb35cefee841d0881b08c456df5a97 already in progress: creating tablet
I20260320 15:27:49.452135 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.452476 28447 raft_consensus.cc:2468] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:49.453017 28240 leader_election.cc:304] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.453451 30773 raft_consensus.cc:2804] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.453538 30773 raft_consensus.cc:493] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.453629 30773 raft_consensus.cc:3060] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.453794 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:49.454280 28239 leader_election.cc:343] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.456449 30773 raft_consensus.cc:515] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.457149 30773 leader_election.cc:290] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.458917 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.459126 28437 raft_consensus.cc:3060] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.459133 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:49.459683 28239 leader_election.cc:343] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.451141 29640 rpcz_store.cc:276] 0320 15:27:41.535797 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.535872 (+ 75us) service_pool.cc:225] Handling call
0320 15:27:49.450952 (+7915080us) ts_tablet_manager.cc:1151] Deleting tablet ac4c7dc6859048ce956927201df44a86
0320 15:27:49.450965 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.451033 (+ 68us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7914817}
W20260320 15:27:49.460355 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ac4c7dc6859048ce956927201df44a86 because the tablet was not found. No further retry: Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:49.460758 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 243) took 7872 ms. Trace:
I20260320 15:27:49.460844 29640 rpcz_store.cc:276] 0320 15:27:41.588591 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.588664 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:49.460651 (+7871987us) ts_tablet_manager.cc:1151] Deleting tablet b2355eadcae6483bae7ae475d74dd131
0320 15:27:49.460663 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.460739 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7871739}
I20260320 15:27:49.461187 30840 log.cc:1199] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/4190329df1b644c392ff8cbace9aae7a
W20260320 15:27:49.461323 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b2355eadcae6483bae7ae475d74dd131 because the tablet was not found. No further retry: Not found: Tablet not found: b2355eadcae6483bae7ae475d74dd131
I20260320 15:27:49.461549 30840 ts_tablet_manager.cc:1950] T 4190329df1b644c392ff8cbace9aae7a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:49.462314 28862 consensus_peers.cc:597] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 -> Peer 07e8bb8819eb4cb4892e0cba1f758892 (127.27.74.61:44739): Couldn't send request to peer 07e8bb8819eb4cb4892e0cba1f758892. Status: Network error: Client connection negotiation failed: client connection to 127.27.74.61:44739: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.462407 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 244) took 7800 ms. Trace:
I20260320 15:27:49.462477 29640 rpcz_store.cc:276] 0320 15:27:41.662177 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.662260 (+ 83us) service_pool.cc:225] Handling call
0320 15:27:49.462304 (+7800044us) ts_tablet_manager.cc:1151] Deleting tablet 7394e941c99e4358ad7a33db84b15348
0320 15:27:49.462316 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.462388 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7799696}
W20260320 15:27:49.463016 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7394e941c99e4358ad7a33db84b15348 because the tablet was not found. No further retry: Not found: Tablet not found: 7394e941c99e4358ad7a33db84b15348
I20260320 15:27:49.463076 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]) successfully deleted
W20260320 15:27:49.463617 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 49654c67435a48448794879464f6cfb2 because the tablet was not found. No further retry: Not found: Tablet not found: 49654c67435a48448794879464f6cfb2
I20260320 15:27:49.463843 28734 tablet_bootstrap.cc:654] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.464604 28736 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.464949 28736 ts_tablet_manager.cc:1403] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.000s
W20260320 15:27:49.465312 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 4cfad5cf843a437e8e15c7236d1dc5fe because the tablet was not found. No further retry: Not found: Tablet not found: 4cfad5cf843a437e8e15c7236d1dc5fe
W20260320 15:27:49.465806 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7840b529206d482283cc550d2c5e7606 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7840b529206d482283cc550d2c5e7606 already in progress: creating tablet
I20260320 15:27:49.465756 28736 raft_consensus.cc:359] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.465968 28736 raft_consensus.cc:385] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.466032 28736 raft_consensus.cc:740] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.466439 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 245) took 7803 ms. Trace:
I20260320 15:27:49.466542 29640 rpcz_store.cc:276] 0320 15:27:41.663457 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.663522 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:49.466313 (+7802791us) ts_tablet_manager.cc:1151] Deleting tablet 26818e620ff649d18b671e4f4c7c7e74
0320 15:27:49.466327 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.466413 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7802543}
W20260320 15:27:49.467089 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 26818e620ff649d18b671e4f4c7c7e74 because the tablet was not found. No further retry: Not found: Tablet not found: 26818e620ff649d18b671e4f4c7c7e74
I20260320 15:27:49.414324 28906 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_1bbba63ee33c407f8d02e8f63df7245e" } new_table_name: "default.Table_fdf1117756d04248ad30f6a0c9101c6f" modify_external_catalogs: true
I20260320 15:27:49.467584 28736 consensus_queue.cc:260] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.468976 28437 raft_consensus.cc:2468] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:49.469364 28901 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_af7bda6fe8e14815bfd5bf848ac65d4c" } modify_external_catalogs: true
I20260320 15:27:49.469684 28901 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_af7bda6fe8e14815bfd5bf848ac65d4c" } modify_external_catalogs: true
I20260320 15:27:49.469834 28900 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_76bddc5536ff43e895889c5c30c5068f" } new_table_name: "default.Table_7ac4015d018a4c8e96ea949d122786a3" modify_external_catalogs: true
I20260320 15:27:49.469462 28889 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_4478d6addfd34e128e575a94d94c09f3" } modify_external_catalogs: true
I20260320 15:27:49.470430 28905 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_283c1ab1e64449abaa1696a2d2bc7911" } modify_external_catalogs: true
I20260320 15:27:49.470520 28889 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_4478d6addfd34e128e575a94d94c09f3" } modify_external_catalogs: true
I20260320 15:27:49.470628 28905 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_283c1ab1e64449abaa1696a2d2bc7911" } modify_external_catalogs: true
I20260320 15:27:49.472532 28904 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_1b61db70cc644eaeb675dfb95960f295" } new_table_name: "default.Table_f57610d7b6234bb9bcc7aa518b2bca0f" modify_external_catalogs: true
I20260320 15:27:49.467789 30840 tablet_replica.cc:333] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.478099 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 246) took 7804 ms. Trace:
I20260320 15:27:49.478217 29640 rpcz_store.cc:276] 0320 15:27:41.664408 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.664476 (+ 68us) service_pool.cc:225] Handling call
0320 15:27:49.468468 (+7803992us) ts_tablet_manager.cc:1151] Deleting tablet a2775196c8e04dcc80c7e089b57579fd
0320 15:27:49.468480 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.468572 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7803812}
W20260320 15:27:49.478762 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet a2775196c8e04dcc80c7e089b57579fd because the tablet was not found. No further retry: Not found: Tablet not found: a2775196c8e04dcc80c7e089b57579fd
I20260320 15:27:49.479169 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 247) took 7814 ms. Trace:
I20260320 15:27:49.479262 29640 rpcz_store.cc:276] 0320 15:27:41.665198 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.665254 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:49.479051 (+7813797us) ts_tablet_manager.cc:1151] Deleting tablet 8a18f32c29224bd883ceb20658dff228
0320 15:27:49.479063 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.479149 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7813627}
W20260320 15:27:49.479717 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 8a18f32c29224bd883ceb20658dff228 because the tablet was not found. No further retry: Not found: Tablet not found: 8a18f32c29224bd883ceb20658dff228
I20260320 15:27:49.474135 28240 leader_election.cc:304] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.481385 30840 raft_consensus.cc:2243] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.481617 30840 pending_rounds.cc:70] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:49.481814 30840 pending_rounds.cc:77] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387840786808832 op_type: NO_OP noop_request { }
I20260320 15:27:49.482023 30840 raft_consensus.cc:2889] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260320 15:27:49.482197 30840 raft_consensus.cc:2272] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.481740 30852 raft_consensus.cc:2804] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.482577 30852 raft_consensus.cc:697] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:49.482851 28733 tablet_bootstrap.cc:492] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.483035 28733 ts_tablet_manager.cc:1403] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.178s user 0.025s sys 0.028s
I20260320 15:27:49.482936 30852 consensus_queue.cc:237] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.478001 28736 ts_tablet_manager.cc:1434] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260320 15:27:49.484036 28736 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.470625 29643 ts_tablet_manager.cc:1929] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.484148 28733 raft_consensus.cc:359] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.484333 28733 raft_consensus.cc:385] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.484390 28733 raft_consensus.cc:740] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.484642 29643 log.cc:1199] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/20eec704800e4d8b924d97556cc2700e
I20260320 15:27:49.484609 28733 consensus_queue.cc:260] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.485054 29643 ts_tablet_manager.cc:1950] T 20eec704800e4d8b924d97556cc2700e P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.485292 30840 ts_tablet_manager.cc:1916] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.486320 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 20eec704800e4d8b924d97556cc2700e (table default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]) successfully deleted
I20260320 15:27:49.486757 29643 tablet_replica.cc:333] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.487026 29643 raft_consensus.cc:2243] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.487262 29643 raft_consensus.cc:2272] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.488152 28733 ts_tablet_manager.cc:1434] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:49.489581 28733 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.492511 28733 tablet_bootstrap.cc:654] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.495940 28734 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.496145 28734 ts_tablet_manager.cc:1403] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.081s user 0.002s sys 0.001s
I20260320 15:27:49.496891 28734 raft_consensus.cc:359] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.497128 28734 raft_consensus.cc:385] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.497222 28734 raft_consensus.cc:740] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.497521 28734 consensus_queue.cc:260] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.499303 28734 ts_tablet_manager.cc:1434] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260320 15:27:49.501783 28734 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.502192 28736 tablet_bootstrap.cc:654] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.487927 29643 ts_tablet_manager.cc:1916] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.506261 28734 tablet_bootstrap.cc:654] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.513811 28733 tablet_bootstrap.cc:492] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.513970 28733 ts_tablet_manager.cc:1403] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.003s
I20260320 15:27:49.514990 28733 raft_consensus.cc:359] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.515195 28733 raft_consensus.cc:385] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.515262 28733 raft_consensus.cc:740] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.515435 28733 consensus_queue.cc:260] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.515935 28733 ts_tablet_manager.cc:1434] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:49.516793 28733 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.518527 28733 tablet_bootstrap.cc:654] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.480360 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 248) took 7814 ms. Trace:
I20260320 15:27:49.520565 29640 rpcz_store.cc:276] 0320 15:27:41.666154 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.666212 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:49.480247 (+7814035us) ts_tablet_manager.cc:1151] Deleting tablet 9a072859f39c4c28ae65b0e9afa79d66
0320 15:27:49.480262 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.480340 (+ 78us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7813839}
W20260320 15:27:49.521126 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9a072859f39c4c28ae65b0e9afa79d66 because the tablet was not found. No further retry: Not found: Tablet not found: 9a072859f39c4c28ae65b0e9afa79d66
I20260320 15:27:49.521162 30773 raft_consensus.cc:493] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.521333 30773 raft_consensus.cc:515] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.522040 30773 leader_election.cc:290] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.523079 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.523406 28437 raft_consensus.cc:2468] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:49.524151 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.524986 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 249) took 7847 ms. Trace:
I20260320 15:27:49.525104 29640 rpcz_store.cc:276] 0320 15:27:41.677629 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:41.677691 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:49.524862 (+7847171us) ts_tablet_manager.cc:1151] Deleting tablet ea95b7a1b30e42cebcbfcd551c58f679
0320 15:27:49.524876 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.524962 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7846937}
I20260320 15:27:49.525205 28240 leader_election.cc:304] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.525480 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 251) took 7450 ms. Trace:
I20260320 15:27:49.525553 29640 rpcz_store.cc:276] 0320 15:27:42.075020 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.075114 (+ 94us) service_pool.cc:225] Handling call
0320 15:27:49.525402 (+7450288us) ts_tablet_manager.cc:1151] Deleting tablet b6201b74fdb0428e98f2eebb0b24d01a
0320 15:27:49.525412 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.525466 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7450001}
I20260320 15:27:49.525636 30466 raft_consensus.cc:2804] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.525710 30466 raft_consensus.cc:493] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.525764 30466 raft_consensus.cc:3060] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.525841 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 252) took 7449 ms. Trace:
I20260320 15:27:49.527277 29640 rpcz_store.cc:276] 0320 15:27:42.076372 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.076434 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:49.525768 (+7449334us) ts_tablet_manager.cc:1151] Deleting tablet 583ed053b2a742c7b029f348f4495101
0320 15:27:49.525778 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.525828 (+ 50us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7449056}
I20260320 15:27:49.528208 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 255) took 6985 ms. Trace:
I20260320 15:27:49.528853 30466 raft_consensus.cc:515] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.529649 30466 leader_election.cc:290] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.528308 29640 rpcz_store.cc:276] 0320 15:27:42.542553 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:42.542620 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:49.528040 (+6985420us) ts_tablet_manager.cc:1151] Deleting tablet aba79ddde6e842dfb67597c5d9e371cb
0320 15:27:49.528064 (+ 24us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.528185 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6985195}
I20260320 15:27:49.531369 29640 tablet_replica.cc:333] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.531701 29640 raft_consensus.cc:2243] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.531909 29640 raft_consensus.cc:2272] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.532616 29640 ts_tablet_manager.cc:1916] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.533933 30456 raft_consensus.cc:493] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.534093 30456 raft_consensus.cc:515] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.534520 28733 tablet_bootstrap.cc:492] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.534646 28733 ts_tablet_manager.cc:1403] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.005s
I20260320 15:27:49.534737 30456 leader_election.cc:290] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.535497 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a185f84c24ea402684aa1cc3628d4f11" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.535615 28733 raft_consensus.cc:359] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.535739 28733 raft_consensus.cc:385] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.535785 28733 raft_consensus.cc:740] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.535990 28733 consensus_queue.cc:260] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:49.536080 28382 leader_election.cc:343] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.536356 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a185f84c24ea402684aa1cc3628d4f11" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.536439 28736 tablet_bootstrap.cc:492] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.536605 28736 ts_tablet_manager.cc:1403] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.053s user 0.005s sys 0.004s
I20260320 15:27:49.537315 28736 raft_consensus.cc:359] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.537562 28736 raft_consensus.cc:385] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.537643 28736 raft_consensus.cc:740] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.537889 28733 ts_tablet_manager.cc:1434] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20260320 15:27:49.537937 28383 leader_election.cc:343] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.537889 28736 consensus_queue.cc:260] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.538058 28383 leader_election.cc:304] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.538434 28736 ts_tablet_manager.cc:1434] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:49.538749 30687 raft_consensus.cc:2749] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.538910 28736 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.539394 28733 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.540012 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.540205 28447 raft_consensus.cc:3060] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.541355 28733 tablet_bootstrap.cc:654] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.541857 30840 ts_tablet_manager.cc:1929] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.542842 28447 raft_consensus.cc:2468] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
W20260320 15:27:49.546794 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ea95b7a1b30e42cebcbfcd551c58f679 because the tablet was not found. No further retry: Not found: Tablet not found: ea95b7a1b30e42cebcbfcd551c58f679
I20260320 15:27:49.546845 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:49.547142 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b6201b74fdb0428e98f2eebb0b24d01a because the tablet was not found. No further retry: Not found: Tablet not found: b6201b74fdb0428e98f2eebb0b24d01a
W20260320 15:27:49.547379 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 583ed053b2a742c7b029f348f4495101 because the tablet was not found. No further retry: Not found: Tablet not found: 583ed053b2a742c7b029f348f4495101
W20260320 15:27:49.547868 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet aba79ddde6e842dfb67597c5d9e371cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba79ddde6e842dfb67597c5d9e371cb already in progress: creating tablet
I20260320 15:27:49.549801 29643 ts_tablet_manager.cc:1929] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.550870 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 24f3b75e30f34eb7a1f3799c5536fd76 (DEFAULT_TABLE table=default.Table_7b38daf0b39d41f18e3f8a65deca4870 [id=8a7270bf9b384e6f8a50cdcc937e0643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.551333 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f3b75e30f34eb7a1f3799c5536fd76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.544063 28240 leader_election.cc:304] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.552063 30773 raft_consensus.cc:2804] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.552214 30773 raft_consensus.cc:697] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:49.552516 30773 consensus_queue.cc:237] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:49.545969 28239 leader_election.cc:343] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.557098 28239 leader_election.cc:343] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.557149 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 24f3b75e30f34eb7a1f3799c5536fd76 (DEFAULT_TABLE table=default.Table_7b38daf0b39d41f18e3f8a65deca4870 [id=8a7270bf9b384e6f8a50cdcc937e0643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.557582 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f3b75e30f34eb7a1f3799c5536fd76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.562333 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 755ebf335ed84318ba5ac1530a186edf (DEFAULT_TABLE table=default.Table_1c80a258a5e34d8b9708d39f47c0fdd8 [id=5eeffd3493dc437885bafde61f122830]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:49.562505 28240 consensus_peers.cc:597] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.562844 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ebf335ed84318ba5ac1530a186edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.550756 28734 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.565050 28734 ts_tablet_manager.cc:1403] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.000s
I20260320 15:27:49.555927 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 24f3b75e30f34eb7a1f3799c5536fd76 (DEFAULT_TABLE table=default.Table_7b38daf0b39d41f18e3f8a65deca4870 [id=8a7270bf9b384e6f8a50cdcc937e0643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.566203 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f3b75e30f34eb7a1f3799c5536fd76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.566107 28734 raft_consensus.cc:359] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.566442 28734 raft_consensus.cc:385] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.566545 28734 raft_consensus.cc:740] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.566848 28734 consensus_queue.cc:260] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.568843 30840 log.cc:1199] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/20eec704800e4d8b924d97556cc2700e
I20260320 15:27:49.570595 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:49.571893 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 755ebf335ed84318ba5ac1530a186edf (DEFAULT_TABLE table=default.Table_1c80a258a5e34d8b9708d39f47c0fdd8 [id=5eeffd3493dc437885bafde61f122830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.572310 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ebf335ed84318ba5ac1530a186edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.572554 28734 ts_tablet_manager.cc:1434] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260320 15:27:49.575439 28734 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.577759 29643 log.cc:1199] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/197f1935099140a98c5eb801cf44046a
I20260320 15:27:49.578305 28734 tablet_bootstrap.cc:654] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.578524 29643 ts_tablet_manager.cc:1950] T 197f1935099140a98c5eb801cf44046a P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.580143 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 197f1935099140a98c5eb801cf44046a (table default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]) successfully deleted
I20260320 15:27:49.581326 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 755ebf335ed84318ba5ac1530a186edf (DEFAULT_TABLE table=default.Table_1c80a258a5e34d8b9708d39f47c0fdd8 [id=5eeffd3493dc437885bafde61f122830]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:49.581604 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
I20260320 15:27:49.581979 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 755ebf335ed84318ba5ac1530a186edf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:49.582250 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 06cb44b9cc6445cf8324cf52190d6773 because the tablet was not found. No further retry: Not found: Tablet not found: 06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:49.582573 29643 tablet_replica.cc:333] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.582784 29643 raft_consensus.cc:2243] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.582947 29643 raft_consensus.cc:2272] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.583794 29643 ts_tablet_manager.cc:1916] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.597900 28736 tablet_bootstrap.cc:654] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.598593 28734 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.598664 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:49.599057 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:49.599416 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:49.599076 28734 ts_tablet_manager.cc:1403] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260320 15:27:49.599375 30840 ts_tablet_manager.cc:1950] T 20eec704800e4d8b924d97556cc2700e P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.599998 28294 raft_consensus.cc:3060] T 11bef661385b4221a991710ffb34dccb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.600063 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:49.600404 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:49.600593 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:49.600831 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:49.601006 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:49.601351 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:49.601951 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:49.602135 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:49.602326 28902 catalog_manager.cc:5671] T 2cb7cbe0a9224412880144927ec8fa99 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.602998 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 20eec704800e4d8b924d97556cc2700e (table default.Table_2926e23005b34db8a1d7e340b1a1e55e [id=52ed203e1d7c4b7a976289c124fd5868]) successfully deleted
I20260320 15:27:49.603091 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:49.606354 28734 raft_consensus.cc:359] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.607950 28297 raft_consensus.cc:3060] T 514e7b02a28142c2b371644089044a9d P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:49.609195 28239 consensus_peers.cc:597] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 197f1935099140a98c5eb801cf44046a. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.612808 30395 consensus_queue.cc:1048] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.621816 28734 raft_consensus.cc:385] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.622017 28734 raft_consensus.cc:740] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.621934 29640 ts_tablet_manager.cc:1929] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.624475 29640 log.cc:1199] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:49.624823 29640 ts_tablet_manager.cc:1950] T 2e9791ac61b34fc09a8292c79f0c4032 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.625137 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 256) took 6500 ms. Trace:
I20260320 15:27:49.625227 29640 rpcz_store.cc:276] 0320 15:27:43.125090 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.125140 (+ 50us) service_pool.cc:225] Handling call
0320 15:27:49.531300 (+6406160us) ts_tablet_manager.cc:1151] Deleting tablet 2e9791ac61b34fc09a8292c79f0c4032
0320 15:27:49.531315 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.618711 (+ 87396us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.621817 (+ 3106us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.625120 (+ 3303us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6405941}
I20260320 15:27:49.625906 29640 tablet_replica.cc:333] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.626096 29640 raft_consensus.cc:2243] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.626268 29640 raft_consensus.cc:2272] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.626848 29640 ts_tablet_manager.cc:1916] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.628983 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
W20260320 15:27:49.629835 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because the tablet was not found. No further retry: Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
I20260320 15:27:49.622393 28734 consensus_queue.cc:260] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:49.630432 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 90cd18e2ccbf48c8b804329bd3abf847 because the tablet was not found. No further retry: Not found: Tablet not found: 90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:49.630602 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 2e9791ac61b34fc09a8292c79f0c4032 (table default.Table_3c050d31cc0248c7beb57aafd767d53f [id=3c0f4e2c474b4bd086c799306bd302f1]) successfully deleted
I20260320 15:27:49.631340 28416 tablet_service.cc:1511] Processing CreateTablet for tablet f404ba3026104c6785b58149fe165bc1 (DEFAULT_TABLE table=default.Table_39290b96eadb43968da86872c4682a5d [id=231c8996ce884a509dfca80419f9b99f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.631728 28556 tablet_service.cc:1511] Processing CreateTablet for tablet e1536f2e36344fac8343ef04d19d678e (DEFAULT_TABLE table=default.Table_c9ff393f07814790b10d6c0faef3d408 [id=a2177c1c57154d75a626ee2a6d013ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.631872 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f404ba3026104c6785b58149fe165bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.632076 28734 ts_tablet_manager.cc:1434] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20260320 15:27:49.632107 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1536f2e36344fac8343ef04d19d678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.633204 30687 raft_consensus.cc:493] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.633345 30687 raft_consensus.cc:515] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.634083 30687 leader_election.cc:290] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.634994 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.630628 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:49.635643 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:49.635682 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 3ccbf4f32636463f8e1e005a629b92cc because the tablet was not found. No further retry: Not found: Tablet not found: 3ccbf4f32636463f8e1e005a629b92cc
I20260320 15:27:49.636009 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
W20260320 15:27:49.636018 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet d823ab41194542fba5540e53e9f4680b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d823ab41194542fba5540e53e9f4680b already in progress: creating tablet
W20260320 15:27:49.636201 28382 leader_election.cc:343] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.636595 28734 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:49.636785 28383 leader_election.cc:343] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.636893 28383 leader_election.cc:304] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.637648 28268 tablet_service.cc:1511] Processing CreateTablet for tablet e1536f2e36344fac8343ef04d19d678e (DEFAULT_TABLE table=default.Table_c9ff393f07814790b10d6c0faef3d408 [id=a2177c1c57154d75a626ee2a6d013ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.637784 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:49.638132 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1536f2e36344fac8343ef04d19d678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.638604 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:49.638870 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:49.639106 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:49.639587 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:49.640319 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:49.640614 30456 raft_consensus.cc:2749] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.640713 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:49.640995 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:49.641597 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:49.642393 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 408f5b4d67514838a17c180ae2cd71df (table default.Table_1671adae9cbe4d76a53e9544d76d5497 [id=b2921bb259f24e1682ec25e2f083ff5f]). Expected version 1 got 0
I20260320 15:27:49.643016 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:49.643281 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:49.643553 28425 tablet_service.cc:1511] Processing CreateTablet for tablet e1536f2e36344fac8343ef04d19d678e (DEFAULT_TABLE table=default.Table_c9ff393f07814790b10d6c0faef3d408 [id=a2177c1c57154d75a626ee2a6d013ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.643635 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_8ef5be02dff04c058181c86aa2de56a0 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 4 got 0
I20260320 15:27:49.643999 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1536f2e36344fac8343ef04d19d678e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.644342 30655 raft_consensus.cc:493] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.644457 28902 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:49.644491 30655 raft_consensus.cc:515] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.645093 30655 leader_election.cc:290] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.646188 28281 tablet_service.cc:1511] Processing CreateTablet for tablet f404ba3026104c6785b58149fe165bc1 (DEFAULT_TABLE table=default.Table_39290b96eadb43968da86872c4682a5d [id=231c8996ce884a509dfca80419f9b99f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260320 15:27:49.646409 28524 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:49.646776 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f404ba3026104c6785b58149fe165bc1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:49.646814 28524 leader_election.cc:343] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: ae3422cf33324ab89a2b44e27cae0eb2
I20260320 15:27:49.646919 28524 leader_election.cc:304] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.649937 30395 raft_consensus.cc:2749] T ae3422cf33324ab89a2b44e27cae0eb2 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.650120 30395 raft_consensus.cc:493] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.650223 30395 raft_consensus.cc:515] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.650846 30395 leader_election.cc:290] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.651672 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.652117 30773 raft_consensus.cc:493] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.652277 30773 raft_consensus.cc:515] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.652904 30773 leader_election.cc:290] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.653673 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:49.654309 28239 leader_election.cc:343] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.654685 28524 leader_election.cc:343] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.655203 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.655382 28447 raft_consensus.cc:2468] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:49.655858 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.656000 28447 raft_consensus.cc:2468] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:49.656585 28240 leader_election.cc:304] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.657220 30773 raft_consensus.cc:2804] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.657325 30773 raft_consensus.cc:493] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.657433 30773 raft_consensus.cc:3060] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.660957 28524 leader_election.cc:304] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:49.661283 30655 raft_consensus.cc:2804] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.661398 30655 raft_consensus.cc:493] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.661499 30655 raft_consensus.cc:3060] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.663800 30395 consensus_queue.cc:1048] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.664090 30655 raft_consensus.cc:515] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.664656 30655 leader_election.cc:290] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.665766 28736 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.665938 28736 ts_tablet_manager.cc:1403] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.127s user 0.002s sys 0.002s
I20260320 15:27:49.666903 28736 raft_consensus.cc:359] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.667223 28736 raft_consensus.cc:385] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.667755 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.667949 28437 raft_consensus.cc:3060] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:49.668269 28239 consensus_peers.cc:597] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.669454 30840 tablet_replica.cc:333] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.669772 30840 raft_consensus.cc:2243] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.669982 30840 raft_consensus.cc:2272] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.670552 28562 tablet_service.cc:1511] Processing CreateTablet for tablet f404ba3026104c6785b58149fe165bc1 (DEFAULT_TABLE table=default.Table_39290b96eadb43968da86872c4682a5d [id=231c8996ce884a509dfca80419f9b99f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:49.670861 28437 raft_consensus.cc:2468] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:49.667248 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
W20260320 15:27:49.672050 28524 leader_election.cc:343] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.672391 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f404ba3026104c6785b58149fe165bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:49.671607 28733 tablet_bootstrap.cc:492] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.672685 28733 ts_tablet_manager.cc:1403] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.133s user 0.007s sys 0.000s
I20260320 15:27:49.673763 30687 raft_consensus.cc:493] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.673910 30687 raft_consensus.cc:515] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.674531 30687 leader_election.cc:290] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.675777 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.675441 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:49.676579 28383 leader_election.cc:343] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.676999 28382 leader_election.cc:343] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.677004 28733 raft_consensus.cc:359] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.677114 28382 leader_election.cc:304] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.677315 28733 raft_consensus.cc:385] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.677419 28733 raft_consensus.cc:740] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
W20260320 15:27:49.677932 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6d9b0f1936f54c699211c5ee9709f9ed because the tablet was not found. No further retry: Not found: Tablet not found: 6d9b0f1936f54c699211c5ee9709f9ed
W20260320 15:27:49.678237 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 87cb35cefee841d0881b08c456df5a97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87cb35cefee841d0881b08c456df5a97 already in progress: creating tablet
I20260320 15:27:49.678233 28733 consensus_queue.cc:260] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:49.678550 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec already in progress: creating tablet
I20260320 15:27:49.679287 30456 raft_consensus.cc:2749] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.679481 30456 raft_consensus.cc:493] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.679617 30456 raft_consensus.cc:515] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.680217 30456 leader_election.cc:290] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.680918 28733 ts_tablet_manager.cc:1434] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260320 15:27:49.681143 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.681980 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:49.682411 28382 leader_election.cc:343] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.682538 28383 leader_election.cc:343] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.682669 28383 leader_election.cc:304] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.683410 30456 raft_consensus.cc:2749] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.681612 28524 leader_election.cc:304] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c
I20260320 15:27:49.683879 30395 raft_consensus.cc:2804] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.684007 30395 raft_consensus.cc:697] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:49.684233 30395 consensus_queue.cc:237] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.685665 29640 ts_tablet_manager.cc:1929] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.685783 29640 log.cc:1199] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/06cb44b9cc6445cf8324cf52190d6773
I20260320 15:27:49.686141 29640 ts_tablet_manager.cc:1950] T 06cb44b9cc6445cf8324cf52190d6773 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.686507 30629 raft_consensus.cc:493] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.686599 30840 ts_tablet_manager.cc:1916] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.686643 30629 raft_consensus.cc:515] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.687224 30629 leader_election.cc:290] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:49.688882 28524 leader_election.cc:343] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 4190329df1b644c392ff8cbace9aae7a
W20260320 15:27:49.689232 28524 leader_election.cc:343] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 4190329df1b644c392ff8cbace9aae7a
I20260320 15:27:49.689339 28524 leader_election.cc:304] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.689587 30655 raft_consensus.cc:2749] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.689684 30773 raft_consensus.cc:515] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.690846 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.691073 28437 raft_consensus.cc:3060] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.693107 28733 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.693320 30840 ts_tablet_manager.cc:1929] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.693492 30840 log.cc:1199] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/db16d86664794305bde714a7ff4d5caf
I20260320 15:27:49.693876 30840 ts_tablet_manager.cc:1950] T db16d86664794305bde714a7ff4d5caf P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
W20260320 15:27:49.694866 28524 consensus_peers.cc:597] T 28c453eaf49c4171b0f93d3b26690d83 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28c453eaf49c4171b0f93d3b26690d83. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.695501 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet db16d86664794305bde714a7ff4d5caf (table default.Table_0c66e258345e4633aea4d4962f922159 [id=b662362d91fe4864be94d1f750830c2e]) successfully deleted
W20260320 15:27:49.695894 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 87a3d5e4ff8f42409482af49c636be0c because the tablet was not found. No further retry: Not found: Tablet not found: 87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:49.667469 28736 raft_consensus.cc:740] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
W20260320 15:27:49.696156 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8d92e17cea044541968356ae993be1e7 because the tablet was not found. No further retry: Not found: Tablet not found: 8d92e17cea044541968356ae993be1e7
W20260320 15:27:49.696509 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 35347116811e4aee8142a503dd21843a because the tablet was not found. No further retry: Not found: Tablet not found: 35347116811e4aee8142a503dd21843a
I20260320 15:27:49.696435 28736 consensus_queue.cc:260] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:49.696987 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 54266dd376164673bffe829dd515b83d because the tablet was not found. No further retry: Not found: Tablet not found: 54266dd376164673bffe829dd515b83d
W20260320 15:27:49.697741 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet ac4c7dc6859048ce956927201df44a86 because the tablet was not found. No further retry: Not found: Tablet not found: ac4c7dc6859048ce956927201df44a86
I20260320 15:27:49.699028 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:49.699671 28736 ts_tablet_manager.cc:1434] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.034s user 0.003s sys 0.000s
I20260320 15:27:49.699860 28575 raft_consensus.cc:3060] T 54c75b9deab44c829f6720480e38c80b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.700014 28736 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.701764 28736 tablet_bootstrap.cc:654] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.704077 30456 raft_consensus.cc:493] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.704234 30456 raft_consensus.cc:515] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.704862 30456 leader_election.cc:290] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.705368 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16ffb28029e4b7ca37ce36efe9e0271" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:49.705920 28239 leader_election.cc:343] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:49.706902 28382 leader_election.cc:343] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.708163 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16ffb28029e4b7ca37ce36efe9e0271" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.708205 30587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1caebf640d4ec0bf9b28c4f496ca18" member_type: VOTER last_known_addr { host: "127.27.74.62" port: 43951 } } peers { permanent_uuid: "07e8bb8819eb4cb4892e0cba1f758892" member_type: VOTER last_known_addr { host: "127.27.74.61" port: 44739 } } peers { permanent_uuid: "336af1a126ca4ab583c03527b890239a" member_type: VOTER last_known_addr { host: "127.27.74.60" port: 42563 } } }
I20260320 15:27:49.708436 30587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18 [sys.catalog]: This master's current role is: LEADER
W20260320 15:27:49.708777 28383 leader_election.cc:343] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.708932 28383 leader_election.cc:304] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.709221 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 269) took 5911 ms. Trace:
I20260320 15:27:49.709324 29640 rpcz_store.cc:276] 0320 15:27:43.797840 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.802562 (+ 4722us) service_pool.cc:225] Handling call
0320 15:27:49.625851 (+5823289us) ts_tablet_manager.cc:1151] Deleting tablet 06cb44b9cc6445cf8324cf52190d6773
0320 15:27:49.625862 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.682259 (+ 56397us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.685545 (+ 3286us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.709199 (+ 23654us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5823037}
I20260320 15:27:49.709470 30687 raft_consensus.cc:2749] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.709797 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 06cb44b9cc6445cf8324cf52190d6773 (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]) successfully deleted
I20260320 15:27:49.709860 30773 leader_election.cc:290] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.711901 28733 tablet_bootstrap.cc:654] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.714998 28437 raft_consensus.cc:2468] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:49.715510 29640 tablet_replica.cc:333] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.716909 28240 leader_election.cc:304] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.718401 30773 raft_consensus.cc:2804] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.718638 30773 raft_consensus.cc:697] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:49.718926 30773 consensus_queue.cc:237] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.720852 29640 raft_consensus.cc:2243] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.721282 29640 raft_consensus.cc:2272] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.722121 29640 ts_tablet_manager.cc:1916] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.728899 28733 tablet_bootstrap.cc:492] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.729063 28733 ts_tablet_manager.cc:1403] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
W20260320 15:27:49.736918 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b16ffb28029e4b7ca37ce36efe9e0271 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b16ffb28029e4b7ca37ce36efe9e0271 already in progress: creating tablet
I20260320 15:27:49.736979 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:49.738307 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:49.738263 28733 raft_consensus.cc:359] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.738466 28733 raft_consensus.cc:385] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.738518 28733 raft_consensus.cc:740] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.738752 28733 consensus_queue.cc:260] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.739665 28892 catalog_manager.cc:5671] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.740378 30840 tablet_replica.cc:333] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.740698 30840 raft_consensus.cc:2243] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.741072 30629 consensus_queue.cc:1048] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.741205 30840 pending_rounds.cc:70] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Trying to abort 1 pending ops.
I20260320 15:27:49.741300 30840 pending_rounds.cc:77] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7266387838803329024 op_type: NO_OP noop_request { }
I20260320 15:27:49.741644 30840 raft_consensus.cc:2889] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260320 15:27:49.740136 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet cc97a5f2bd964a05bb7b2427226debdd because the tablet was not found. No further retry: Not found: Tablet not found: cc97a5f2bd964a05bb7b2427226debdd
W20260320 15:27:49.742429 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fcff253b9cce45b4ad11dc38397afee2 because the tablet was not found. No further retry: Not found: Tablet not found: fcff253b9cce45b4ad11dc38397afee2
W20260320 15:27:49.742753 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet b4cad900411e4c9686701aa99206dd23 because the tablet was not found. No further retry: Not found: Tablet not found: b4cad900411e4c9686701aa99206dd23
I20260320 15:27:49.742992 30840 raft_consensus.cc:2272] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.744387 29640 ts_tablet_manager.cc:1929] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.744673 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:49.745365 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:49.746941 30687 raft_consensus.cc:493] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.747193 29640 log.cc:1199] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/1bd3032950254e6f9db8da35dbb1acc5
I20260320 15:27:49.747149 30687 raft_consensus.cc:515] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.747773 29640 ts_tablet_manager.cc:1950] T 1bd3032950254e6f9db8da35dbb1acc5 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.747983 30687 leader_election.cc:290] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.748360 30840 ts_tablet_manager.cc:1916] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.748793 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:49.749094 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ab44039124f36ac60601098586b4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.749405 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 270) took 5939 ms. Trace:
I20260320 15:27:49.749486 29640 rpcz_store.cc:276] 0320 15:27:43.809538 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:43.809594 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:49.715429 (+5905835us) ts_tablet_manager.cc:1151] Deleting tablet 1bd3032950254e6f9db8da35dbb1acc5
0320 15:27:49.715445 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.733460 (+ 18015us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.744253 (+ 10793us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.749385 (+ 5132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5905525}
W20260320 15:27:49.749795 28383 leader_election.cc:343] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.749970 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 1bd3032950254e6f9db8da35dbb1acc5 (table default.Table_f4751ac3d9fd417ba487d086d653b85d [id=1c228777a1f446808e13303615920653]) successfully deleted
I20260320 15:27:49.750401 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:49.750715 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:49.751787 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 278) took 5732 ms. Trace:
I20260320 15:27:49.752322 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:49.752486 29640 rpcz_store.cc:276] 0320 15:27:44.019845 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.019910 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:49.751638 (+5731728us) ts_tablet_manager.cc:1151] Deleting tablet 09c87a35c8fd4d87a1cd9e5ca3db15ec
0320 15:27:49.751651 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.751767 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5731527}
I20260320 15:27:49.752918 29643 ts_tablet_manager.cc:1929] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.753580 29640 tablet_replica.cc:333] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.753909 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:49.759779 29640 raft_consensus.cc:2243] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:49.760401 28239 consensus_peers.cc:597] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c -> Peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Couldn't send request to peer 80636bb608a1431b955576e296ff63bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:49.760772 29640 raft_consensus.cc:2272] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.761456 29640 ts_tablet_manager.cc:1916] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.760319 28892 catalog_manager.cc:5671] T da29d2ffcb07428789c22a573d680289 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.763841 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:49.764153 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:49.766496 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:49.766639 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d04ab44039124f36ac60601098586b4e" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.766999 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:49.767028 30840 ts_tablet_manager.cc:1929] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.767179 30840 log.cc:1199] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/197f1935099140a98c5eb801cf44046a
I20260320 15:27:49.767702 30840 ts_tablet_manager.cc:1950] T 197f1935099140a98c5eb801cf44046a P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.767838 28733 ts_tablet_manager.cc:1434] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260320 15:27:49.768041 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:49.768177 28733 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.769042 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 197f1935099140a98c5eb801cf44046a (table default.Table_00ee6904094c4b5abf6e50334f698c40 [id=5fe729f8b4904ef9af804736bb178ae6]) successfully deleted
W20260320 15:27:49.769512 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 28c453eaf49c4171b0f93d3b26690d83 because the tablet was not found. No further retry: Not found: Tablet not found: 28c453eaf49c4171b0f93d3b26690d83
W20260320 15:27:49.769726 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 1bd3032950254e6f9db8da35dbb1acc5 because the tablet was not found. No further retry: Not found: Tablet not found: 1bd3032950254e6f9db8da35dbb1acc5
W20260320 15:27:49.769942 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 8241a342054240d2ac62b729cc9c9a62 because the tablet was not found. No further retry: Not found: Tablet not found: 8241a342054240d2ac62b729cc9c9a62
W20260320 15:27:49.770378 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 56fa691a26114657ad768545a30dfd3c because the tablet was not found. No further retry: Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c
W20260320 15:27:49.770617 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet dd315b5d49104d679b03515b65ee30ac because the tablet was not found. No further retry: Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
W20260320 15:27:49.770805 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 7f94140a5e8b4a06890bca86456d64a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f94140a5e8b4a06890bca86456d64a6 already in progress: creating tablet
I20260320 15:27:49.770833 30840 tablet_replica.cc:333] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
W20260320 15:27:49.771004 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6049ed1c2d84494893bfa20a9d563838 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6049ed1c2d84494893bfa20a9d563838 already in progress: creating tablet
I20260320 15:27:49.771068 30840 raft_consensus.cc:2243] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260320 15:27:49.771198 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 71ec63fb080545de838803033f204fab because the tablet was not found. No further retry: Not found: Tablet not found: 71ec63fb080545de838803033f204fab
I20260320 15:27:49.771306 30840 raft_consensus.cc:2272] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:49.771406 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 6292904782fd4b619483135e58b66684 because the tablet was not found. No further retry: Not found: Tablet not found: 6292904782fd4b619483135e58b66684
I20260320 15:27:49.771610 28733 tablet_bootstrap.cc:654] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:49.771632 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet fa5618796f1a4ff3ab4c1209dad0aede because the tablet was not found. No further retry: Not found: Tablet not found: fa5618796f1a4ff3ab4c1209dad0aede
W20260320 15:27:49.771826 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 16e31e11048b494ebe1fd4d89488b0cf because the tablet was not found. No further retry: Not found: Tablet not found: 16e31e11048b494ebe1fd4d89488b0cf
W20260320 15:27:49.772006 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 5a20d7f22cfa403184e73b24d565b60a because the tablet was not found. No further retry: Not found: Tablet not found: 5a20d7f22cfa403184e73b24d565b60a
W20260320 15:27:49.772238 28873 catalog_manager.cc:4969] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because the tablet was not found. No further retry: Not found: Tablet not found: cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:49.773191 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:49.778260 30655 raft_consensus.cc:493] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:49.778402 30655 raft_consensus.cc:515] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.778975 30655 leader_election.cc:290] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
W20260320 15:27:49.779946 28524 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
W20260320 15:27:49.780134 28524 leader_election.cc:343] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:49.780206 28524 leader_election.cc:304] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.780354 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:49.780676 30655 raft_consensus.cc:2749] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:49.780769 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:49.781014 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:49.784404 28736 tablet_bootstrap.cc:492] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.784639 28736 ts_tablet_manager.cc:1403] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.085s user 0.009s sys 0.009s
I20260320 15:27:49.785347 30840 ts_tablet_manager.cc:1916] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.788663 28736 raft_consensus.cc:359] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.788933 28736 raft_consensus.cc:385] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.789126 28736 raft_consensus.cc:740] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.789407 28736 consensus_queue.cc:260] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.753044 29643 log.cc:1199] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/a185f84c24ea402684aa1cc3628d4f11
W20260320 15:27:49.771365 28382 leader_election.cc:343] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:49.790932 28736 ts_tablet_manager.cc:1434] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260320 15:27:49.790949 28382 leader_election.cc:304] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:49.792163 30687 raft_consensus.cc:2749] T d04ab44039124f36ac60601098586b4e P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.792238 28736 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.764477 28734 tablet_bootstrap.cc:654] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.792309 28892 catalog_manager.cc:5671] T 547d82cee19b4959b3a779c8b8088e83 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.794090 28736 tablet_bootstrap.cc:654] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.793126 28892 catalog_manager.cc:5671] T 514e7b02a28142c2b371644089044a9d P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.794644 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:49.795665 29640 ts_tablet_manager.cc:1929] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:49.796857 29640 log.cc:1199] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:49.795200 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:49.797555 29640 ts_tablet_manager.cc:1950] T 87a3d5e4ff8f42409482af49c636be0c P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.797820 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:49.798630 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:49.799592 30840 ts_tablet_manager.cc:1929] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.799731 30840 log.cc:1199] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/5de834725bda4f249882914f1247ab62
I20260320 15:27:49.800140 30840 ts_tablet_manager.cc:1950] T 5de834725bda4f249882914f1247ab62 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.800558 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]). Expected version 1 got 0
I20260320 15:27:49.801048 30840 tablet_replica.cc:333] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.801308 30840 raft_consensus.cc:2243] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.801514 30840 raft_consensus.cc:2272] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.802218 29643 ts_tablet_manager.cc:1950] T a185f84c24ea402684aa1cc3628d4f11 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.799650 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 286) took 5062 ms. Trace:
I20260320 15:27:49.801367 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]) successfully deleted
I20260320 15:27:49.802629 29640 rpcz_store.cc:276] 0320 15:27:44.736958 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.737287 (+ 329us) service_pool.cc:225] Handling call
0320 15:27:49.753524 (+5016237us) ts_tablet_manager.cc:1151] Deleting tablet 87a3d5e4ff8f42409482af49c636be0c
0320 15:27:49.753536 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.773660 (+ 20124us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.795521 (+ 21861us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.799630 (+ 4109us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5016052}
I20260320 15:27:49.803342 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:49.804131 29640 tablet_replica.cc:333] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:49.804872 30395 raft_consensus.cc:493] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:49.805012 30395 raft_consensus.cc:515] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.805691 30395 leader_election.cc:290] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.803869 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 87a3d5e4ff8f42409482af49c636be0c (table default.Table_2cf33331acf14700b31b2edd8666f285 [id=363e365bac174b28be76afaf6ad44d2e]) successfully deleted
I20260320 15:27:49.804975 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet a185f84c24ea402684aa1cc3628d4f11 (table default.Table_8b5b70bee34f4caabd4a276b20f57793 [id=e4f2c5eb7fff4a718a8cdbbb8b72d953]) successfully deleted
W20260320 15:27:49.806380 28524 leader_election.cc:343] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 6d9b0f1936f54c699211c5ee9709f9ed
W20260320 15:27:49.806489 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 4190329df1b644c392ff8cbace9aae7a because the tablet was not found. No further retry: Not found: Tablet not found: 4190329df1b644c392ff8cbace9aae7a
W20260320 15:27:49.806715 28524 leader_election.cc:343] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 6d9b0f1936f54c699211c5ee9709f9ed
W20260320 15:27:49.806752 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6771bd6a3f0e47d99a0191eae25cb2d2 because the tablet was not found. No further retry: Not found: Tablet not found: 6771bd6a3f0e47d99a0191eae25cb2d2
I20260320 15:27:49.806824 28524 leader_election.cc:304] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
W20260320 15:27:49.806944 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet aba79ddde6e842dfb67597c5d9e371cb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba79ddde6e842dfb67597c5d9e371cb already in progress: creating tablet
W20260320 15:27:49.807142 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 1bd3032950254e6f9db8da35dbb1acc5 because the tablet was not found. No further retry: Not found: Tablet not found: 1bd3032950254e6f9db8da35dbb1acc5
I20260320 15:27:49.807178 30395 raft_consensus.cc:2749] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260320 15:27:49.807368 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet d823ab41194542fba5540e53e9f4680b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d823ab41194542fba5540e53e9f4680b already in progress: creating tablet
W20260320 15:27:49.807648 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 7a4b5a658cd84b04b50b54973c065889 because the tablet was not found. No further retry: Not found: Tablet not found: 7a4b5a658cd84b04b50b54973c065889
I20260320 15:27:49.808334 28734 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.808535 29640 raft_consensus.cc:2243] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.808745 29640 raft_consensus.cc:2272] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.809527 28734 ts_tablet_manager.cc:1403] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.173s user 0.003s sys 0.001s
I20260320 15:27:49.810333 30629 raft_consensus.cc:493] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.810462 30629 raft_consensus.cc:515] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.811022 30629 leader_election.cc:290] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.811318 30866 consensus_queue.cc:1048] T b20efd94b09746c88bb48ba36af4759e P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.812383 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.820254 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:49.821302 29640 ts_tablet_manager.cc:1916] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.822999 28892 catalog_manager.cc:5671] T 594357400f8c44138c7c4f11bf6d5afd P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:49.829310 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcf4e017e2e743d7b38628d6f1e0d72e" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.830672 28302 raft_consensus.cc:2393] T bcf4e017e2e743d7b38628d6f1e0d72e P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate d22e73471b9046d28d3cf63c905b11b7 in this term.
I20260320 15:27:49.831979 28524 leader_election.cc:304] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.832458 30866 raft_consensus.cc:3060] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.832839 28734 raft_consensus.cc:359] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.833063 28734 raft_consensus.cc:385] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.833125 28734 raft_consensus.cc:740] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.834141 29640 ts_tablet_manager.cc:1929] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.834249 29640 log.cc:1199] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:49.834563 29640 ts_tablet_manager.cc:1950] T e2243e6811744b7cba99a487f8e8bb83 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:49.835271 28734 consensus_queue.cc:260] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.835515 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 408f5b4d67514838a17c180ae2cd71df (table default.Table_1671adae9cbe4d76a53e9544d76d5497 [id=b2921bb259f24e1682ec25e2f083ff5f]). Expected version 1 got 0
I20260320 15:27:49.835522 30866 raft_consensus.cc:2749] T bcf4e017e2e743d7b38628d6f1e0d72e P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.837036 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 418) took 1011 ms. Trace:
I20260320 15:27:49.837172 29643 rpcz_store.cc:276] 0320 15:27:48.825275 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.825874 (+ 599us) service_pool.cc:225] Handling call
0320 15:27:49.836863 (+1010989us) ts_tablet_manager.cc:1151] Deleting tablet 9dbd3b0e92a744dcbc132037c67303ab
0320 15:27:49.836883 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.837006 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":976288}
W20260320 15:27:49.837963 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9dbd3b0e92a744dcbc132037c67303ab because the tablet was not found. No further retry: Not found: Tablet not found: 9dbd3b0e92a744dcbc132037c67303ab
I20260320 15:27:49.837930 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:49.840377 28734 ts_tablet_manager.cc:1434] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.031s user 0.002s sys 0.000s
I20260320 15:27:49.840809 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:49.842971 28733 tablet_bootstrap.cc:492] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.843127 28733 ts_tablet_manager.cc:1403] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.075s user 0.007s sys 0.002s
I20260320 15:27:49.844293 28733 raft_consensus.cc:359] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.844483 28733 raft_consensus.cc:385] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.844606 28733 raft_consensus.cc:740] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.844587 28892 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d78ab2f7e0d7413ea9049dba9cbd8633 (table default.Table_8ef5be02dff04c058181c86aa2de56a0 [id=03aa8e7e0e044e31b29509e7ed1ff617]). Expected version 4 got 0
I20260320 15:27:49.844837 28734 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:49.844853 28733 consensus_queue.cc:260] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.846221 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 419) took 1020 ms. Trace:
I20260320 15:27:49.846335 29643 rpcz_store.cc:276] 0320 15:27:48.825365 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.825965 (+ 600us) service_pool.cc:225] Handling call
0320 15:27:49.846078 (+1020113us) ts_tablet_manager.cc:1151] Deleting tablet 54266dd376164673bffe829dd515b83d
0320 15:27:49.846094 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.846193 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":985473}
I20260320 15:27:49.846576 28889 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet c37fc78b5d1546f69723cac88ce26ea8
I20260320 15:27:49.846751 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 420) took 1021 ms. Trace:
I20260320 15:27:49.846827 29643 rpcz_store.cc:276] 0320 15:27:48.825479 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.826057 (+ 578us) service_pool.cc:225] Handling call
0320 15:27:49.846658 (+1020601us) ts_tablet_manager.cc:1151] Deleting tablet e2243e6811744b7cba99a487f8e8bb83
0320 15:27:49.846669 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.846735 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":986020}
I20260320 15:27:49.847118 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 421) took 1021 ms. Trace:
I20260320 15:27:49.847146 28889 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 185) took 1611 ms. Trace:
I20260320 15:27:49.847185 29643 rpcz_store.cc:276] 0320 15:27:48.825582 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.826180 (+ 598us) service_pool.cc:225] Handling call
0320 15:27:49.847046 (+1020866us) ts_tablet_manager.cc:1151] Deleting tablet 6292904782fd4b619483135e58b66684
0320 15:27:49.847055 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.847105 (+ 50us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":986365}
I20260320 15:27:49.847453 29643 tablet_replica.cc:333] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:49.847719 29643 raft_consensus.cc:2243] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:49.847898 29643 raft_consensus.cc:2272] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.848563 29643 ts_tablet_manager.cc:1916] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.850050 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.850629 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.847272 28889 rpcz_store.cc:276] 0320 15:27:48.235337 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.426231 (+1190894us) service_pool.cc:225] Handling call
0320 15:27:49.469290 (+ 43059us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf7bf6 00007fe6118e7486 00007fe6118e71b5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.470636 (+ 1346us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.470794 (+ 158us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:49.470858 (+ 64us) catalog_manager.cc:2824] Locking tablets
0320 15:27:49.471106 (+ 248us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:49.497783 (+ 26677us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:49.497854 (+ 71us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:49.846491 (+348637us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:49.847101 (+ 610us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.476323 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.476415 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.476421 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:49.476426 (+ 5us) tablet.cc:661] Decoding operations
0320 15:27:49.476543 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.476556 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.476562 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.476652 (+ 90us) tablet.cc:700] Row locks acquired
0320 15:27:49.476655 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.476684 (+ 29us) write_op.cc:270] Start()
0320 15:27:49.476712 (+ 28us) write_op.cc:276] Timestamp: P: 1774020469476650 usec, L: 0
0320 15:27:49.476714 (+ 2us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.476845 (+ 131us) log.cc:844] Serialized 725 byte log entry
0320 15:27:49.495488 (+ 18643us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.496934 (+ 1446us) write_op.cc:301] APPLY: starting
0320 15:27:49.496944 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.496970 (+ 26us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.497026 (+ 56us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.497070 (+ 44us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.497094 (+ 24us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:49.497101 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.497145 (+ 44us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.497177 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.497188 (+ 11us) write_op.cc:454] Released schema lock
0320 15:27:49.497193 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":98240768,"child_traces":[["op",{"apply.queue_time_us":1420,"apply.run_cpu_time_us":315,"apply.run_wall_time_us":314,"num_ops":2,"prepare.queue_time_us":3903,"prepare.run_cpu_time_us":613,"prepare.run_wall_time_us":661,"raft.queue_time_us":29,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":18759,"thread_start_us":40786,"threads_started":1}]]}
I20260320 15:27:49.851002 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet c37fc78b5d1546f69723cac88ce26ea8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:49.855921 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 54266dd376164673bffe829dd515b83d because the tablet was not found. No further retry: Not found: Tablet not found: 54266dd376164673bffe829dd515b83d
W20260320 15:27:49.856216 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet e2243e6811744b7cba99a487f8e8bb83 because the tablet was not found. No further retry: Not found: Tablet not found: e2243e6811744b7cba99a487f8e8bb83
I20260320 15:27:49.856323 30840 ts_tablet_manager.cc:1916] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:49.856448 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6292904782fd4b619483135e58b66684 because the tablet was not found. No further retry: Not found: Tablet not found: 6292904782fd4b619483135e58b66684
I20260320 15:27:49.860178 28736 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.860385 28736 ts_tablet_manager.cc:1403] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.022s
I20260320 15:27:49.860540 28901 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 60ad9b381aaa47f1afb63df266faee09
I20260320 15:27:49.861099 28901 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 186) took 1620 ms. Trace:
I20260320 15:27:49.861370 28736 raft_consensus.cc:359] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.861593 28736 raft_consensus.cc:385] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.861613 28733 ts_tablet_manager.cc:1434] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260320 15:27:49.861649 28736 raft_consensus.cc:740] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.861212 28901 rpcz_store.cc:276] 0320 15:27:48.241092 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.446902 (+1205810us) service_pool.cc:225] Handling call
0320 15:27:49.469290 (+ 22388us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf7bf6 00007fe6118e7486 00007fe6118e71b5 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.469891 (+ 601us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.470053 (+ 162us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:49.470132 (+ 79us) catalog_manager.cc:2824] Locking tablets
0320 15:27:49.545564 (+ 75432us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:49.560743 (+ 15179us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:49.560867 (+ 124us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:49.860457 (+299590us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:49.861029 (+ 572us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.550655 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.550792 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.550802 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:49.550806 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:49.550963 (+ 157us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.550982 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.550990 (+ 8us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.551135 (+ 145us) tablet.cc:700] Row locks acquired
0320 15:27:49.551140 (+ 5us) write_op.cc:260] PREPARE: finished
0320 15:27:49.551192 (+ 52us) write_op.cc:270] Start()
0320 15:27:49.551240 (+ 48us) write_op.cc:276] Timestamp: P: 1774020469551146 usec, L: 0
0320 15:27:49.551245 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.552166 (+ 921us) log.cc:844] Serialized 725 byte log entry
0320 15:27:49.556898 (+ 4732us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.558828 (+ 1930us) write_op.cc:301] APPLY: starting
0320 15:27:49.558852 (+ 24us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.558913 (+ 61us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.558961 (+ 48us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.559054 (+ 93us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.559103 (+ 49us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:49.559116 (+ 13us) write_op.cc:312] APPLY: finished
0320 15:27:49.559216 (+ 100us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.560152 (+ 936us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.560617 (+ 465us) write_op.cc:454] Released schema lock
0320 15:27:49.560633 (+ 16us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.560946 (+ 313us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.560955 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":50649984,"child_traces":[["op",{"apply.queue_time_us":1855,"apply.run_cpu_time_us":642,"apply.run_wall_time_us":2109,"num_ops":2,"prepare.queue_time_us":4021,"prepare.run_cpu_time_us":1087,"prepare.run_wall_time_us":2028,"raft.queue_time_us":826,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":645,"replication_time_us":5616,"spinlock_wait_cycles":1595136}]]}
I20260320 15:27:49.861881 28736 consensus_queue.cc:260] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.862432 28736 ts_tablet_manager.cc:1434] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:49.863016 28733 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:49.864809 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 60ad9b381aaa47f1afb63df266faee09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.865170 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 60ad9b381aaa47f1afb63df266faee09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.865195 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 60ad9b381aaa47f1afb63df266faee09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.865929 30655 raft_consensus.cc:493] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.866133 30655 raft_consensus.cc:515] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.866873 30655 leader_election.cc:290] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.867522 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.867678 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d87a2f5513684da79f6eebbed89d0392" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.867810 28736 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.867820 28437 raft_consensus.cc:2393] T d87a2f5513684da79f6eebbed89d0392 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:49.868314 28524 leader_election.cc:304] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.868731 30655 raft_consensus.cc:3060] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.869262 28736 tablet_bootstrap.cc:654] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.871084 28734 tablet_bootstrap.cc:654] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.871773 30655 raft_consensus.cc:2749] T d87a2f5513684da79f6eebbed89d0392 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.872694 30773 raft_consensus.cc:493] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.872821 30773 raft_consensus.cc:515] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.873380 30773 leader_election.cc:290] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:49.873957 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.874181 28439 raft_consensus.cc:2393] T 4bd1799035a3431092fbdfad3c42d22b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce5383d28b347ab979765b880e5773c in current term 1: Already voted for candidate 80636bb608a1431b955576e296ff63bb in this term.
I20260320 15:27:49.875337 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bd1799035a3431092fbdfad3c42d22b" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:49.876012 28239 leader_election.cc:304] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c; no voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.876435 30773 raft_consensus.cc:3060] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.876639 28890 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet a1965a7f25ca4c4d95575a6c93ec376f
I20260320 15:27:49.877343 28890 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 182) took 1715 ms. Trace:
I20260320 15:27:49.877487 28890 rpcz_store.cc:276] 0320 15:27:48.161565 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.413779 (+1252214us) service_pool.cc:225] Handling call
0320 15:27:49.414261 (+ 482us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.414435 (+ 174us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:49.414505 (+ 70us) catalog_manager.cc:2824] Locking tablets
0320 15:27:49.414681 (+ 176us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:49.492190 (+ 77509us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:49.492390 (+ 200us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:49.876540 (+384150us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:49.877290 (+ 750us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.417671 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.417816 (+ 145us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.417823 (+ 7us) write_op.cc:435] Acquired schema lock
0320 15:27:49.417826 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:49.417954 (+ 128us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.417968 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.417974 (+ 6us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.418082 (+ 108us) tablet.cc:700] Row locks acquired
0320 15:27:49.418084 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.418120 (+ 36us) write_op.cc:270] Start()
0320 15:27:49.418153 (+ 33us) write_op.cc:276] Timestamp: P: 1774020469418091 usec, L: 0
0320 15:27:49.418157 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.418322 (+ 165us) log.cc:844] Serialized 725 byte log entry
0320 15:27:49.488094 (+ 69772us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.488574 (+ 480us) write_op.cc:301] APPLY: starting
0320 15:27:49.488594 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.491611 (+ 3017us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.491678 (+ 67us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.491795 (+ 117us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.491833 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:49.491841 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:49.491951 (+ 110us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.492071 (+ 120us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.492093 (+ 22us) write_op.cc:454] Released schema lock
0320 15:27:49.492100 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.493617 (+ 1517us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.493627 (+ 10us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":386,"apply.run_cpu_time_us":699,"apply.run_wall_time_us":5025,"num_ops":2,"prepare.queue_time_us":2134,"prepare.run_cpu_time_us":739,"prepare.run_wall_time_us":740,"replication_time_us":69901}]]}
I20260320 15:27:49.878623 28890 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_467ca42537f0421bacf771620a950e06" } new_table_name: "default.Table_a975717748634ce98154a00bf30e6c33" modify_external_catalogs: true
I20260320 15:27:49.879334 30773 raft_consensus.cc:2749] T 4bd1799035a3431092fbdfad3c42d22b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.880179 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.880630 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:49.880955 28904 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ab9893da469449bba556d11bd347d89b consensus state has no leader
W20260320 15:27:49.880961 28900 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8ac11b0ca134c948fdd9ff4b114e6aa consensus state has no leader
I20260320 15:27:49.883499 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet a1965a7f25ca4c4d95575a6c93ec376f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.884853 30655 consensus_queue.cc:1048] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.886961 29643 ts_tablet_manager.cc:1929] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:49.887099 29643 log.cc:1199] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/b16ffb28029e4b7ca37ce36efe9e0271
I20260320 15:27:49.887425 29643 ts_tablet_manager.cc:1950] T b16ffb28029e4b7ca37ce36efe9e0271 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:49.887785 28733 tablet_bootstrap.cc:654] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.891290 28905 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet bf37dbb8bc8e48c5a11f106ecd93dd78
I20260320 15:27:49.891965 28905 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 187) took 1598 ms. Trace:
I20260320 15:27:49.892234 30687 consensus_queue.cc:1048] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.892069 28905 rpcz_store.cc:276] 0320 15:27:48.293501 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.470259 (+1176758us) service_pool.cc:225] Handling call
0320 15:27:49.470724 (+ 465us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.470858 (+ 134us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:49.470908 (+ 50us) catalog_manager.cc:2824] Locking tablets
0320 15:27:49.471120 (+ 212us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:49.506944 (+ 35824us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:49.507049 (+ 105us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:49.891190 (+384141us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:49.891902 (+ 712us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.474418 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.474524 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.474530 (+ 6us) write_op.cc:435] Acquired schema lock
0320 15:27:49.474532 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.474666 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.474680 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.474684 (+ 4us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:49.474798 (+ 114us) tablet.cc:700] Row locks acquired
0320 15:27:49.474801 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.474838 (+ 37us) write_op.cc:270] Start()
0320 15:27:49.474868 (+ 30us) write_op.cc:276] Timestamp: P: 1774020469474802 usec, L: 0
0320 15:27:49.474872 (+ 4us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.475028 (+ 156us) log.cc:844] Serialized 756 byte log entry
0320 15:27:49.495406 (+ 20378us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.496181 (+ 775us) write_op.cc:301] APPLY: starting
0320 15:27:49.496194 (+ 13us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.496234 (+ 40us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.496254 (+ 20us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.496306 (+ 52us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.496331 (+ 25us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:49.496337 (+ 6us) write_op.cc:312] APPLY: finished
0320 15:27:49.496387 (+ 50us) log.cc:844] Serialized 29 byte log entry
0320 15:27:49.496425 (+ 38us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496437 (+ 12us) write_op.cc:454] Released schema lock
0320 15:27:49.496442 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.496501 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496505 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":745,"apply.run_cpu_time_us":314,"apply.run_wall_time_us":313,"num_ops":2,"prepare.queue_time_us":2611,"prepare.run_cpu_time_us":739,"prepare.run_wall_time_us":826,"raft.queue_time_us":81,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":20520}]]}
I20260320 15:27:49.892935 30840 ts_tablet_manager.cc:1929] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:49.893042 30840 log.cc:1199] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/d5095b42b69d40539118c1f63a95f795
I20260320 15:27:49.892926 28905 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_fe5422b8425b4fe9988570705b9fddcd" } new_table_name: "default.Table_2bb51a781fe145748a5db64e26c3a085" modify_external_catalogs: true
I20260320 15:27:49.893394 30840 ts_tablet_manager.cc:1950] T d5095b42b69d40539118c1f63a95f795 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:49.894179 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.894569 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.895257 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:49 UTC) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.900647 30629 raft_consensus.cc:493] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.900839 30629 raft_consensus.cc:515] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.901449 30629 leader_election.cc:290] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.903859 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.905161 30840 tablet_replica.cc:333] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:49.905474 30840 raft_consensus.cc:2243] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:49.905879 30840 raft_consensus.cc:2272] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:49.906656 30840 ts_tablet_manager.cc:1916] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:49.908097 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6240d441c624b238f7c13eb3c5fa9fc" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.908197 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet d5095b42b69d40539118c1f63a95f795 (table default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]) successfully deleted
I20260320 15:27:49.908767 28524 leader_election.cc:304] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:49.909379 30395 raft_consensus.cc:2749] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:49.916483 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 422) took 1089 ms. Trace:
I20260320 15:27:49.916625 29643 rpcz_store.cc:276] 0320 15:27:48.826654 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.826707 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:49.847400 (+1020693us) ts_tablet_manager.cc:1151] Deleting tablet b16ffb28029e4b7ca37ce36efe9e0271
0320 15:27:49.847410 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.883621 (+ 36211us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.886846 (+ 3225us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.916453 (+ 29607us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":986656}
I20260320 15:27:49.917387 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet b16ffb28029e4b7ca37ce36efe9e0271 (table default.Table_9b2e32517f9d49a9a117be78ffddb91e [id=5df94a1a6fc04a4983dc2888ef98c2d6]) successfully deleted
I20260320 15:27:49.918066 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 423) took 1088 ms. Trace:
I20260320 15:27:49.918193 29643 rpcz_store.cc:276] 0320 15:27:48.829158 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.829259 (+ 101us) service_pool.cc:225] Handling call
0320 15:27:49.917891 (+1088632us) ts_tablet_manager.cc:1151] Deleting tablet db16d86664794305bde714a7ff4d5caf
0320 15:27:49.917907 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.918041 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1057105}
I20260320 15:27:49.918548 28900 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 184) took 1744 ms. Trace:
W20260320 15:27:49.918771 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet db16d86664794305bde714a7ff4d5caf because the tablet was not found. No further retry: Not found: Tablet not found: db16d86664794305bde714a7ff4d5caf
I20260320 15:27:49.919919 27944 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskc3SESu/build/asan/bin/kudu
/tmp/dist-test-taskc3SESu/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.74.61:44739
--webserver_interface=127.27.74.61
--webserver_port=39103
--builtin_ntp_servers=127.27.74.20:43127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.74.62:43951,127.27.74.61:44739,127.27.74.60:42563
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260320 15:27:49.920262 28904 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 188) took 1586 ms. Trace:
I20260320 15:27:49.920825 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 424) took 1091 ms. Trace:
I20260320 15:27:49.920933 29643 rpcz_store.cc:276] 0320 15:27:48.829299 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.829464 (+ 165us) service_pool.cc:225] Handling call
0320 15:27:49.920680 (+1091216us) ts_tablet_manager.cc:1151] Deleting tablet 87a3d5e4ff8f42409482af49c636be0c
0320 15:27:49.920699 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.920799 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1059842}
I20260320 15:27:49.920390 28904 rpcz_store.cc:276] 0320 15:27:48.333701 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.471857 (+1138156us) service_pool.cc:225] Handling call
0320 15:27:49.472715 (+ 858us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.472949 (+ 234us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:49.473157 (+ 208us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:49.473240 (+ 83us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:49.555891 (+ 82651us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:49.920175 (+364284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.477002 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.477090 (+ 88us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.477094 (+ 4us) write_op.cc:435] Acquired schema lock
0320 15:27:49.477097 (+ 3us) tablet.cc:661] Decoding operations
0320 15:27:49.477175 (+ 78us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.477184 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.477188 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:49.477242 (+ 54us) tablet.cc:700] Row locks acquired
0320 15:27:49.477244 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.477268 (+ 24us) write_op.cc:270] Start()
0320 15:27:49.477291 (+ 23us) write_op.cc:276] Timestamp: P: 1774020469477237 usec, L: 0
0320 15:27:49.477294 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.477412 (+ 118us) log.cc:844] Serialized 405 byte log entry
0320 15:27:49.536472 (+ 59060us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.539880 (+ 3408us) write_op.cc:301] APPLY: starting
0320 15:27:49.539901 (+ 21us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.539948 (+ 47us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.539974 (+ 26us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.540029 (+ 55us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.540064 (+ 35us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:49.540072 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:49.540159 (+ 87us) log.cc:844] Serialized 23 byte log entry
0320 15:27:49.543525 (+ 3366us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.554419 (+ 10894us) write_op.cc:454] Released schema lock
0320 15:27:49.554427 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.554667 (+ 240us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.554673 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3324,"apply.run_cpu_time_us":563,"apply.run_wall_time_us":14778,"num_ops":1,"prepare.queue_time_us":3332,"prepare.run_cpu_time_us":492,"prepare.run_wall_time_us":541,"raft.queue_time_us":31,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":59138}]]}
W20260320 15:27:49.921583 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 87a3d5e4ff8f42409482af49c636be0c because the tablet was not found. No further retry: Not found: Tablet not found: 87a3d5e4ff8f42409482af49c636be0c
I20260320 15:27:49.925006 28734 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.925158 28734 ts_tablet_manager.cc:1403] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.080s user 0.007s sys 0.003s
I20260320 15:27:49.925809 28906 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:33394 (request call id 183) took 1764 ms. Trace:
I20260320 15:27:49.925922 28906 rpcz_store.cc:276] 0320 15:27:48.161746 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.414146 (+1252400us) service_pool.cc:225] Handling call
0320 15:27:49.467409 (+ 53263us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.467694 (+ 285us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:49.467912 (+ 218us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:49.467996 (+ 84us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:49.496129 (+ 28133us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:49.925740 (+429611us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.468641 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.468809 (+ 168us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.468819 (+ 10us) write_op.cc:435] Acquired schema lock
0320 15:27:49.468821 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.468929 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.468942 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.468947 (+ 5us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:49.469019 (+ 72us) tablet.cc:700] Row locks acquired
0320 15:27:49.469022 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:49.473990 (+ 4968us) write_op.cc:270] Start()
0320 15:27:49.474043 (+ 53us) write_op.cc:276] Timestamp: P: 1774020469469024 usec, L: 0
0320 15:27:49.474048 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.474262 (+ 214us) log.cc:844] Serialized 405 byte log entry
0320 15:27:49.495345 (+ 21083us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.495697 (+ 352us) write_op.cc:301] APPLY: starting
0320 15:27:49.495714 (+ 17us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.495754 (+ 40us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.495784 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.495835 (+ 51us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.495874 (+ 39us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:49.495882 (+ 8us) write_op.cc:312] APPLY: finished
0320 15:27:49.495955 (+ 73us) log.cc:844] Serialized 23 byte log entry
0320 15:27:49.496020 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496036 (+ 16us) write_op.cc:454] Released schema lock
0320 15:27:49.496042 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.496115 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496118 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":306,"apply.run_cpu_time_us":407,"apply.run_wall_time_us":407,"num_ops":1,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":830,"prepare.run_wall_time_us":5747,"raft.queue_time_us":826,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":21263}]]}
I20260320 15:27:49.926357 28734 raft_consensus.cc:359] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.918668 28900 rpcz_store.cc:276] 0320 15:27:48.174006 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.424936 (+1250930us) service_pool.cc:225] Handling call
0320 15:27:49.469761 (+ 44825us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x6160000a0954. stack: 00007fe60708f076 00007fe60708ee30 00007fe6060493ad 0000000000986fb4 0000000000986f7d 00007fe6163cf6bd 00007fe6163cf7b3 00007fe612f91b21 00007fe61ac6ac9d 00007fe61adf8e96 00007fe6118e9146 00007fe6118e8e75 00007fe60e04de3e 00007fe60e04cd1e 00007fe60e052f55 00007fe60e05387f
0320 15:27:49.469987 (+ 226us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:49.470197 (+ 210us) catalog_manager.cc:3725] Apply alter schema
0320 15:27:49.470382 (+ 185us) catalog_manager.cc:3757] Acquired catalog manager lock
0320 15:27:49.471760 (+ 1378us) catalog_manager.cc:3915] Updating metadata on disk
0320 15:27:49.497695 (+ 25935us) catalog_manager.cc:3946] Committing alterations to in-memory state
0320 15:27:49.918467 (+420772us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:49.475284 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:49.475390 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:49.475395 (+ 5us) write_op.cc:435] Acquired schema lock
0320 15:27:49.475397 (+ 2us) tablet.cc:661] Decoding operations
0320 15:27:49.475485 (+ 88us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:49.475495 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:49.475499 (+ 4us) tablet.cc:684] Acquiring locks for 1 operations
0320 15:27:49.475623 (+ 124us) tablet.cc:700] Row locks acquired
0320 15:27:49.475625 (+ 2us) write_op.cc:260] PREPARE: finished
0320 15:27:49.475661 (+ 36us) write_op.cc:270] Start()
0320 15:27:49.475688 (+ 27us) write_op.cc:276] Timestamp: P: 1774020469475624 usec, L: 0
0320 15:27:49.475691 (+ 3us) op_driver.cc:348] REPLICATION: starting
0320 15:27:49.476111 (+ 420us) log.cc:844] Serialized 405 byte log entry
0320 15:27:49.495449 (+ 19338us) op_driver.cc:464] REPLICATION: finished
0320 15:27:49.496560 (+ 1111us) write_op.cc:301] APPLY: starting
0320 15:27:49.496570 (+ 10us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:49.496607 (+ 37us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:49.496637 (+ 30us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:49.496669 (+ 32us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:49.496690 (+ 21us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0320 15:27:49.496697 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:49.496750 (+ 53us) log.cc:844] Serialized 23 byte log entry
0320 15:27:49.496785 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496795 (+ 10us) write_op.cc:454] Released schema lock
0320 15:27:49.496800 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:49.496879 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:49.496883 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":102412032,"child_traces":[["op",{"apply.queue_time_us":1083,"apply.run_cpu_time_us":307,"apply.run_wall_time_us":288,"num_ops":1,"prepare.queue_time_us":3101,"prepare.run_cpu_time_us":797,"prepare.run_wall_time_us":1001,"raft.queue_time_us":34,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":19745,"spinlock_wait_cycles":285568}]]}
I20260320 15:27:49.926591 28734 raft_consensus.cc:385] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.926656 28734 raft_consensus.cc:740] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.926844 28900 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_4afe8b5be24444de80274a2c43ad204b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:49.927065 28734 consensus_queue.cc:260] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:49.927510 28900 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4afe8b5be24444de80274a2c43ad204b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.929247 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet 55bd68c6e4364a038a815d1fc05ad25e from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:49.929621 28734 ts_tablet_manager.cc:1434] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260320 15:27:49.926839 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_fa5403c78a5f4823959f74237f673da6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.930244 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa5403c78a5f4823959f74237f673da6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.930814 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 425) took 1101 ms. Trace:
I20260320 15:27:49.930894 29643 rpcz_store.cc:276] 0320 15:27:48.829401 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.829614 (+ 213us) service_pool.cc:225] Handling call
0320 15:27:49.930698 (+1101084us) ts_tablet_manager.cc:1151] Deleting tablet cc97a5f2bd964a05bb7b2427226debdd
0320 15:27:49.930710 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.930794 (+ 84us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1069821}
I20260320 15:27:49.931070 28734 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:49.931385 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet cc97a5f2bd964a05bb7b2427226debdd because the tablet was not found. No further retry: Not found: Tablet not found: cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:49.932723 28734 tablet_bootstrap.cc:654] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.933153 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 426) took 1092 ms. Trace:
I20260320 15:27:49.933578 29643 rpcz_store.cc:276] 0320 15:27:48.840560 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.840633 (+ 73us) service_pool.cc:225] Handling call
0320 15:27:49.933038 (+1092405us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:49.933050 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.933132 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1071091}
I20260320 15:27:49.934185 30866 raft_consensus.cc:493] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:49.934360 30866 raft_consensus.cc:515] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.935122 30866 leader_election.cc:290] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:49.940150 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6d9b0f1936f54c699211c5ee9709f9ed because the tablet was not found. No further retry: Not found: Tablet not found: 6d9b0f1936f54c699211c5ee9709f9ed
I20260320 15:27:49.940855 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 433) took 1077 ms. Trace:
I20260320 15:27:49.942487 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:49.943079 29643 rpcz_store.cc:276] 0320 15:27:48.863831 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.863888 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:49.940681 (+1076793us) ts_tablet_manager.cc:1151] Deleting tablet 87cb35cefee841d0881b08c456df5a97
0320 15:27:49.940697 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.940829 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1076580}
W20260320 15:27:49.944255 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 87cb35cefee841d0881b08c456df5a97 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 87cb35cefee841d0881b08c456df5a97 already in progress: creating tablet
I20260320 15:27:49.942129 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:49.944638 28437 raft_consensus.cc:2468] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:49.944855 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 434) took 1080 ms. Trace:
I20260320 15:27:49.944939 29643 rpcz_store.cc:276] 0320 15:27:48.864213 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.864251 (+ 38us) service_pool.cc:225] Handling call
0320 15:27:49.944737 (+1080486us) ts_tablet_manager.cc:1151] Deleting tablet 35347116811e4aee8142a503dd21843a
0320 15:27:49.944749 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.944835 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1080313}
W20260320 15:27:49.945456 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 35347116811e4aee8142a503dd21843a because the tablet was not found. No further retry: Not found: Tablet not found: 35347116811e4aee8142a503dd21843a
I20260320 15:27:49.946339 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 435) took 1081 ms. Trace:
I20260320 15:27:49.946441 29643 rpcz_store.cc:276] 0320 15:27:48.864561 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.864607 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:49.946234 (+1081627us) ts_tablet_manager.cc:1151] Deleting tablet 2e9791ac61b34fc09a8292c79f0c4032
0320 15:27:49.946244 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.946319 (+ 75us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1081490}
W20260320 15:27:49.947046 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 2e9791ac61b34fc09a8292c79f0c4032 because the tablet was not found. No further retry: Not found: Tablet not found: 2e9791ac61b34fc09a8292c79f0c4032
I20260320 15:27:49.947463 28734 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:49.947649 28734 ts_tablet_manager.cc:1403] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:49.948375 28734 raft_consensus.cc:359] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.948561 28734 raft_consensus.cc:385] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.948618 28734 raft_consensus.cc:740] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.949085 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 437) took 1084 ms. Trace:
I20260320 15:27:49.949554 29643 rpcz_store.cc:276] 0320 15:27:48.865016 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.865062 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:49.948954 (+1083892us) ts_tablet_manager.cc:1151] Deleting tablet cea3bf14653f4555acfed1577a79ee4f
0320 15:27:49.948968 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.949065 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1083752}
W20260320 15:27:49.950604 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet cea3bf14653f4555acfed1577a79ee4f because the tablet was not found. No further retry: Not found: Tablet not found: cea3bf14653f4555acfed1577a79ee4f
I20260320 15:27:49.952203 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:49.952464 28304 raft_consensus.cc:2468] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:49.953614 28524 leader_election.cc:304] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters:
I20260320 15:27:49.953939 30866 raft_consensus.cc:2804] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:49.954027 30866 raft_consensus.cc:493] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:49.954095 30866 raft_consensus.cc:3060] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.955039 30456 consensus_queue.cc:1048] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.955390 28906 master_service.cc:954] ReplaceTablet: received request to replace tablet 9dd293a68f894cbfa122d53918816c28 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:49.956923 30866 raft_consensus.cc:515] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.957932 30866 leader_election.cc:290] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:49.959228 28900 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d71bcddc2804450dbb050fdc6c54338a" } modify_external_catalogs: true
I20260320 15:27:49.959477 28900 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_d71bcddc2804450dbb050fdc6c54338a" } modify_external_catalogs: true
I20260320 15:27:49.961176 30773 consensus_queue.cc:1048] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:49.962889 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:49.963120 28439 raft_consensus.cc:3060] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.966533 28439 raft_consensus.cc:2468] T 76379bf0fa554820be020f647582a72b P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:49.967835 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76379bf0fa554820be020f647582a72b" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:49.967959 28524 leader_election.cc:304] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:49.968062 28304 raft_consensus.cc:3060] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:49.971212 28304 raft_consensus.cc:2468] T 76379bf0fa554820be020f647582a72b P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:49.971700 30395 raft_consensus.cc:2804] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:49.971854 30395 raft_consensus.cc:697] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:49.972100 30395 consensus_queue.cc:237] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:49.977623 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 55bd68c6e4364a038a815d1fc05ad25e
I20260320 15:27:49.978271 28904 catalog_manager.cc:6741] ReplaceTablet: tablet 55bd68c6e4364a038a815d1fc05ad25e deleted and replaced by tablet b58cc1c7766c48cc8829f399afa7146f
I20260320 15:27:49.978818 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_1eeb969c59d948078955219bd038ce5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:49.979892 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1eeb969c59d948078955219bd038ce5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.981503 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e with delete_type TABLET_DATA_DELETED (replaced by tablet b58cc1c7766c48cc8829f399afa7146f) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:49.982072 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e with delete_type TABLET_DATA_DELETED (replaced by tablet b58cc1c7766c48cc8829f399afa7146f) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:49.982133 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 55bd68c6e4364a038a815d1fc05ad25e with delete_type TABLET_DATA_DELETED (replaced by tablet b58cc1c7766c48cc8829f399afa7146f) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:49.983039 28736 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.983201 28736 ts_tablet_manager.cc:1403] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.115s user 0.001s sys 0.008s
I20260320 15:27:49.953713 28734 consensus_queue.cc:260] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.984155 28736 raft_consensus.cc:359] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.984354 28736 raft_consensus.cc:385] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.984419 28736 raft_consensus.cc:740] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.984685 28736 consensus_queue.cc:260] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.985584 28736 ts_tablet_manager.cc:1434] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:49.985889 28736 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.987682 28736 tablet_bootstrap.cc:654] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:49.992054 28736 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:49.992230 28736 ts_tablet_manager.cc:1403] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260320 15:27:49.992959 28736 raft_consensus.cc:359] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:49.993147 28736 raft_consensus.cc:385] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:49.993206 28736 raft_consensus.cc:740] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:49.993177 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e96bc88284294560a8ef0fc23a1b34aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:49.993420 28736 consensus_queue.cc:260] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:49.993899 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e96bc88284294560a8ef0fc23a1b34aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:49.994001 28736 ts_tablet_manager.cc:1434] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:49.994313 28736 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:49.996970 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 427) took 1156 ms. Trace:
I20260320 15:27:49.997103 29643 rpcz_store.cc:276] 0320 15:27:48.840665 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.859748 (+ 19083us) service_pool.cc:225] Handling call
0320 15:27:49.996835 (+1137087us) ts_tablet_manager.cc:1151] Deleting tablet b4cad900411e4c9686701aa99206dd23
0320 15:27:49.996851 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.996943 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1131569}
W20260320 15:27:49.997694 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet b4cad900411e4c9686701aa99206dd23 because the tablet was not found. No further retry: Not found: Tablet not found: b4cad900411e4c9686701aa99206dd23
I20260320 15:27:49.998132 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 414) took 1173 ms. Trace:
I20260320 15:27:49.998446 28733 tablet_bootstrap.cc:492] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:49.998597 28733 ts_tablet_manager.cc:1403] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.136s user 0.005s sys 0.001s
I20260320 15:27:49.999339 28733 raft_consensus.cc:359] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.002481 30840 ts_tablet_manager.cc:1929] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.002661 29643 rpcz_store.cc:276] 0320 15:27:48.824845 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.824894 (+ 49us) service_pool.cc:225] Handling call
0320 15:27:49.997995 (+1173101us) ts_tablet_manager.cc:1151] Deleting tablet 27ccc693e7704650bd77e0c4ca11862e
0320 15:27:49.998010 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.998109 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1132689}
W20260320 15:27:50.003326 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 27ccc693e7704650bd77e0c4ca11862e because the tablet was not found. No further retry: Not found: Tablet not found: 27ccc693e7704650bd77e0c4ca11862e
I20260320 15:27:50.003818 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 436) took 1138 ms. Trace:
I20260320 15:27:50.003957 29643 rpcz_store.cc:276] 0320 15:27:48.864849 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.864889 (+ 40us) service_pool.cc:225] Handling call
0320 15:27:50.003710 (+1138821us) ts_tablet_manager.cc:1151] Deleting tablet 5a20d7f22cfa403184e73b24d565b60a
0320 15:27:50.003720 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.003796 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1138311}
W20260320 15:27:50.004452 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 5a20d7f22cfa403184e73b24d565b60a because the tablet was not found. No further retry: Not found: Tablet not found: 5a20d7f22cfa403184e73b24d565b60a
I20260320 15:27:50.004848 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 438) took 1139 ms. Trace:
I20260320 15:27:50.004973 29643 rpcz_store.cc:276] 0320 15:27:48.865661 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.865708 (+ 47us) service_pool.cc:225] Handling call
0320 15:27:50.004761 (+1139053us) ts_tablet_manager.cc:1151] Deleting tablet 3ed5d220ec4b406aa5bae970f785de63
0320 15:27:50.004770 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.004831 (+ 61us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1138894}
I20260320 15:27:50.005663 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 439) took 1139 ms. Trace:
I20260320 15:27:50.005805 29643 rpcz_store.cc:276] 0320 15:27:48.865865 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.865968 (+ 103us) service_pool.cc:225] Handling call
0320 15:27:50.005578 (+1139610us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:50.005588 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.005647 (+ 59us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1139438}
W20260320 15:27:50.006489 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 41da3cb06b1147fbae2695bc5c9ffaa5 because the tablet was not found. No further retry: Not found: Tablet not found: 41da3cb06b1147fbae2695bc5c9ffaa5
I20260320 15:27:50.006963 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 440) took 1140 ms. Trace:
I20260320 15:27:50.007251 29643 rpcz_store.cc:276] 0320 15:27:48.866072 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.866163 (+ 91us) service_pool.cc:225] Handling call
0320 15:27:50.006858 (+1140695us) ts_tablet_manager.cc:1151] Deleting tablet fcff253b9cce45b4ad11dc38397afee2
0320 15:27:50.006870 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.006942 (+ 72us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1140589}
W20260320 15:27:50.012022 28873 catalog_manager.cc:4969] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet fcff253b9cce45b4ad11dc38397afee2 because the tablet was not found. No further retry: Not found: Tablet not found: fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:50.013793 29643 tablet_replica.cc:333] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:50.014775 29643 raft_consensus.cc:2243] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.015192 29643 raft_consensus.cc:2272] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.015260 30866 consensus_queue.cc:1048] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.018518 28733 raft_consensus.cc:385] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.018635 28733 raft_consensus.cc:740] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.019747 28906 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 9dd293a68f894cbfa122d53918816c28
I20260320 15:27:50.019711 28733 consensus_queue.cc:260] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.020648 28906 catalog_manager.cc:6741] ReplaceTablet: tablet 9dd293a68f894cbfa122d53918816c28 deleted and replaced by tablet d6f07e96754b4c9bb593c1c487162ddf
I20260320 15:27:50.021236 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dd293a68f894cbfa122d53918816c28 with delete_type TABLET_DATA_DELETED (replaced by tablet d6f07e96754b4c9bb593c1c487162ddf) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:50.021466 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_c535c1e04d8d45bc986fa72bbcf8babf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:50.022128 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c535c1e04d8d45bc986fa72bbcf8babf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.023449 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dd293a68f894cbfa122d53918816c28 with delete_type TABLET_DATA_DELETED (replaced by tablet d6f07e96754b4c9bb593c1c487162ddf) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:50.025597 30687 raft_consensus.cc:493] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.025806 30687 raft_consensus.cc:515] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.026132 28733 ts_tablet_manager.cc:1434] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.027s user 0.003s sys 0.000s
I20260320 15:27:50.026624 30687 leader_election.cc:290] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.027908 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba79ddde6e842dfb67597c5d9e371cb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.028602 28733 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
W20260320 15:27:50.028959 28382 leader_election.cc:343] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.033448 28906 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_e9e7782bae064c90a600dabc73647497"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:50.034292 28906 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9e7782bae064c90a600dabc73647497 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.039779 28733 tablet_bootstrap.cc:654] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.041478 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet 0e3f6d9a528946908891f77f951d23dd from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:50.055271 30949 tablet.cc:1723] T 7bb680bb0863485aaad85a749221e81b P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.057303 28734 ts_tablet_manager.cc:1434] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.110s user 0.002s sys 0.000s
I20260320 15:27:50.057698 28734 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.073590 28734 tablet_bootstrap.cc:654] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.083177 30655 raft_consensus.cc:493] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:50.083494 30655 raft_consensus.cc:515] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } }
I20260320 15:27:50.084254 30655 leader_election.cc:290] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:50.057571 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 285) took 5325 ms. Trace:
I20260320 15:27:50.084651 29640 rpcz_store.cc:276] 0320 15:27:44.731641 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.731706 (+ 65us) service_pool.cc:225] Handling call
0320 15:27:49.804069 (+5072363us) ts_tablet_manager.cc:1151] Deleting tablet e2243e6811744b7cba99a487f8e8bb83
0320 15:27:49.804083 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:49.830534 (+ 26451us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:49.834017 (+ 3483us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.057474 (+223457us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4993593}
I20260320 15:27:50.085151 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet e2243e6811744b7cba99a487f8e8bb83 (table default.Table_213392d2bde147b2aae83c2b1e8eb8ea [id=205a5a7d76b644a6b1d3cd35fd92d356]) successfully deleted
I20260320 15:27:50.085388 28575 raft_consensus.cc:3060] T f6240d441c624b238f7c13eb3c5fa9fc P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.085582 29640 tablet_replica.cc:333] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:50.085809 29640 raft_consensus.cc:2243] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.086000 29640 raft_consensus.cc:2272] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.086570 29640 ts_tablet_manager.cc:1916] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.087205 30951 tablet.cc:1723] T 7bb680bb0863485aaad85a749221e81b P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.087292 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 0e3f6d9a528946908891f77f951d23dd
I20260320 15:27:50.088083 28556 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e3f6d9a528946908891f77f951d23dd with delete_type TABLET_DATA_DELETED (replaced by tablet fbfa09813bc7446eabe5c04718fb7731) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:50.089519 28733 tablet_bootstrap.cc:492] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.089720 28733 ts_tablet_manager.cc:1403] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.001s
I20260320 15:27:50.090739 28733 raft_consensus.cc:359] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.021930 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 9dd293a68f894cbfa122d53918816c28 with delete_type TABLET_DATA_DELETED (replaced by tablet d6f07e96754b4c9bb593c1c487162ddf) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:50.027328 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba79ddde6e842dfb67597c5d9e371cb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.092423 28304 raft_consensus.cc:2468] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:50.074750 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb0c26e54e844c7a24367db7e65441b (DEFAULT_TABLE table=default.Table_d65492e9e70b4dcdb7cf91ab5d48531e [id=f881c45434b6453b86b4e1238db7b0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.093037 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb0c26e54e844c7a24367db7e65441b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.075229 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb0c26e54e844c7a24367db7e65441b (DEFAULT_TABLE table=default.Table_d65492e9e70b4dcdb7cf91ab5d48531e [id=f881c45434b6453b86b4e1238db7b0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.095738 28383 leader_election.cc:304] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.096320 30687 raft_consensus.cc:2804] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.096398 30687 raft_consensus.cc:493] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.096493 30687 raft_consensus.cc:3060] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.099210 30687 raft_consensus.cc:515] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.100265 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e3f6d9a528946908891f77f951d23dd with delete_type TABLET_DATA_DELETED (replaced by tablet fbfa09813bc7446eabe5c04718fb7731) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:50.100525 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e3f6d9a528946908891f77f951d23dd with delete_type TABLET_DATA_DELETED (replaced by tablet fbfa09813bc7446eabe5c04718fb7731) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:50.100831 28904 catalog_manager.cc:6741] ReplaceTablet: tablet 0e3f6d9a528946908891f77f951d23dd deleted and replaced by tablet fbfa09813bc7446eabe5c04718fb7731
I20260320 15:27:50.101190 29643 ts_tablet_manager.cc:1916] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.101531 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba79ddde6e842dfb67597c5d9e371cb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:50.101563 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_757caf8aa06441c4838e2e0c6a585dbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:50.102123 28382 leader_election.cc:343] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:50.102206 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_757caf8aa06441c4838e2e0c6a585dbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.103073 30687 leader_election.cc:290] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.103372 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 5fb0c26e54e844c7a24367db7e65441b (DEFAULT_TABLE table=default.Table_d65492e9e70b4dcdb7cf91ab5d48531e [id=f881c45434b6453b86b4e1238db7b0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.103838 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb0c26e54e844c7a24367db7e65441b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.104954 30466 consensus_queue.cc:1048] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:50.105160 28524 leader_election.cc:343] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: b4cad900411e4c9686701aa99206dd23
I20260320 15:27:50.106738 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aba79ddde6e842dfb67597c5d9e371cb" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:50.106963 28304 raft_consensus.cc:3060] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.109726 28304 raft_consensus.cc:2468] T aba79ddde6e842dfb67597c5d9e371cb P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:50.110435 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fb0c26e54e844c7a24367db7e65441b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.112145 28383 leader_election.cc:304] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.112795 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 0371fa6e5ddd4d738cbf92a976646a4c (DEFAULT_TABLE table=default.Table_d24ebbeb63cd41139ef872dfd11b5545 [id=645701c361b048ab820144ffa1274e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.113198 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371fa6e5ddd4d738cbf92a976646a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.113241 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 0371fa6e5ddd4d738cbf92a976646a4c (DEFAULT_TABLE table=default.Table_d24ebbeb63cd41139ef872dfd11b5545 [id=645701c361b048ab820144ffa1274e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.113551 28733 raft_consensus.cc:385] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.113610 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371fa6e5ddd4d738cbf92a976646a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.113624 28733 raft_consensus.cc:740] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.113924 28733 consensus_queue.cc:260] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.116075 28733 ts_tablet_manager.cc:1434] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.026s user 0.016s sys 0.000s
I20260320 15:27:50.117328 28733 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.117812 30954 tablet.cc:1723] T 0acad11980f842e7b3a09216241e493d P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.118942 28733 tablet_bootstrap.cc:654] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.121251 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 992975b892e744bbad9bedddaf63c1a2 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.121721 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 992975b892e744bbad9bedddaf63c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.123301 30963 raft_consensus.cc:493] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.123502 30963 raft_consensus.cc:515] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.123870 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6 (DEFAULT_TABLE table=default.Table_1eeeaa1f75324b11ac9a56307a53ff99 [id=bdf0d6ca83eb4f3697ad0eafdfd5d087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.124251 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.124585 30687 raft_consensus.cc:2804] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.124846 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 945e5c1df3804b358d685bd3e22a716d (DEFAULT_TABLE table=default.Table_cf6e9715cac5401dbfda04f61b2d3ae3 [id=b50d246748014931b5d9b442095880f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.125073 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef0d80754dfd4bbd9817f568063511ef" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.125198 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945e5c1df3804b358d685bd3e22a716d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.125628 28382 leader_election.cc:343] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.125831 30963 leader_election.cc:290] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.086812 30952 tablet.cc:1723] T 7bb680bb0863485aaad85a749221e81b P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.139938 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 945e5c1df3804b358d685bd3e22a716d (DEFAULT_TABLE table=default.Table_cf6e9715cac5401dbfda04f61b2d3ae3 [id=b50d246748014931b5d9b442095880f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.140415 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945e5c1df3804b358d685bd3e22a716d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.140827 28556 tablet_service.cc:1511] Processing CreateTablet for tablet 992975b892e744bbad9bedddaf63c1a2 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.140934 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6 (DEFAULT_TABLE table=default.Table_1eeeaa1f75324b11ac9a56307a53ff99 [id=bdf0d6ca83eb4f3697ad0eafdfd5d087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.141852 30955 tablet.cc:1723] T 0acad11980f842e7b3a09216241e493d P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.142321 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 0371fa6e5ddd4d738cbf92a976646a4c (DEFAULT_TABLE table=default.Table_d24ebbeb63cd41139ef872dfd11b5545 [id=645701c361b048ab820144ffa1274e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.142740 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0371fa6e5ddd4d738cbf92a976646a4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.143824 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 945e5c1df3804b358d685bd3e22a716d (DEFAULT_TABLE table=default.Table_cf6e9715cac5401dbfda04f61b2d3ae3 [id=b50d246748014931b5d9b442095880f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.146425 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 945e5c1df3804b358d685bd3e22a716d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.151795 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.152061 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6 (DEFAULT_TABLE table=default.Table_1eeeaa1f75324b11ac9a56307a53ff99 [id=bdf0d6ca83eb4f3697ad0eafdfd5d087]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.152503 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0ddfd0b8a148c4825f4bf4cfe4b1e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.139938 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 992975b892e744bbad9bedddaf63c1a2 (DEFAULT_TABLE table=default.Table_a2fb8a623b1243d6910bf70670373731 [id=1d8ad4c49a8c442e9285dd628ec1e8ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.155814 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 992975b892e744bbad9bedddaf63c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.156160 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 992975b892e744bbad9bedddaf63c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.163429 30840 log.cc:1199] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/1ccfd2d5e82448b99f1022954b98c6cc
I20260320 15:27:50.163996 30840 ts_tablet_manager.cc:1950] T 1ccfd2d5e82448b99f1022954b98c6cc P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:50.165182 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]) successfully deleted
I20260320 15:27:50.172188 28736 tablet_bootstrap.cc:700] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: Time spent opening tablet: real 0.177s user 0.001s sys 0.000s
I20260320 15:27:50.172665 28736 tablet_bootstrap.cc:654] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:50.176154 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 9d7f7b99899c4001adcbdce58910e717 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d7f7b99899c4001adcbdce58910e717 already in progress: creating tablet
W20260320 15:27:50.179209 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet c37fc78b5d1546f69723cac88ce26ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c37fc78b5d1546f69723cac88ce26ea8 already in progress: creating tablet
W20260320 15:27:50.179600 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 60ad9b381aaa47f1afb63df266faee09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60ad9b381aaa47f1afb63df266faee09 already in progress: creating tablet
W20260320 15:27:50.179915 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet a1965a7f25ca4c4d95575a6c93ec376f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1965a7f25ca4c4d95575a6c93ec376f already in progress: creating tablet
W20260320 15:27:50.181599 28524 leader_election.cc:343] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b4cad900411e4c9686701aa99206dd23
I20260320 15:27:50.181744 28524 leader_election.cc:304] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.182050 30655 raft_consensus.cc:2749] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:50.182153 30840 tablet_replica.cc:333] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:50.182358 30840 raft_consensus.cc:2243] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.182551 30840 raft_consensus.cc:2272] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.183115 30840 ts_tablet_manager.cc:1916] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260320 15:27:50.184664 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf37dbb8bc8e48c5a11f106ecd93dd78 already in progress: creating tablet
W20260320 15:27:50.184995 28873 catalog_manager.cc:4982] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): delete failed for tablet 55bd68c6e4364a038a815d1fc05ad25e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55bd68c6e4364a038a815d1fc05ad25e already in progress: creating tablet
I20260320 15:27:50.185844 28734 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.186033 28734 ts_tablet_manager.cc:1403] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.128s user 0.003s sys 0.000s
I20260320 15:27:50.186859 30687 raft_consensus.cc:697] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:50.187063 28734 raft_consensus.cc:359] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.187194 30687 consensus_queue.cc:237] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:50.188457 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 76340.4us
I20260320 15:27:50.190054 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef0d80754dfd4bbd9817f568063511ef" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.191869 28559 tablet_service.cc:1511] Processing CreateTablet for tablet a06503cd8de14a7cb726271e0b68c436 (DEFAULT_TABLE table=default.Table_d1c4892174e5421194d26b7a9cd25545 [id=d2c045a235bd4b858517a779241c8b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.191987 28272 tablet_service.cc:1511] Processing CreateTablet for tablet a06503cd8de14a7cb726271e0b68c436 (DEFAULT_TABLE table=default.Table_d1c4892174e5421194d26b7a9cd25545 [id=d2c045a235bd4b858517a779241c8b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.192405 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06503cd8de14a7cb726271e0b68c436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.192415 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06503cd8de14a7cb726271e0b68c436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.194128 28276 tablet_service.cc:1511] Processing CreateTablet for tablet 4321cb0619f24a0da475390e9a69491f (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.194500 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4321cb0619f24a0da475390e9a69491f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.197374 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfcd345c77d468b8f5294e46ac18029 (DEFAULT_TABLE table=default.Table_ef3eff1b17b34682941f0ec4b03358ce [id=b7a7d1fadff747e1badfa5c83350a24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.197783 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfcd345c77d468b8f5294e46ac18029. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.198792 28383 leader_election.cc:343] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.198921 28383 leader_election.cc:304] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.187242 28734 raft_consensus.cc:385] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.204632 28734 raft_consensus.cc:740] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.204922 28734 consensus_queue.cc:260] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.205545 30964 raft_consensus.cc:2749] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.208109 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 4321cb0619f24a0da475390e9a69491f (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.208631 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4321cb0619f24a0da475390e9a69491f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.212131 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 4321cb0619f24a0da475390e9a69491f (DEFAULT_TABLE table=default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.212450 28734 ts_tablet_manager.cc:1434] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.026s user 0.001s sys 0.001s
I20260320 15:27:50.212797 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4321cb0619f24a0da475390e9a69491f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.215014 28734 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.216259 30773 raft_consensus.cc:493] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.216436 30773 raft_consensus.cc:515] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.217554 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.217839 28439 raft_consensus.cc:2468] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:50.218331 28240 leader_election.cc:304] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.218611 30852 raft_consensus.cc:2804] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.218674 30852 raft_consensus.cc:493] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.218729 30852 raft_consensus.cc:3060] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.221412 30852 raft_consensus.cc:515] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.222048 30852 leader_election.cc:290] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.223011 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.223284 28444 raft_consensus.cc:3060] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.226850 28444 raft_consensus.cc:2468] T 26ae379304674bc3941af125f0d9d6b3 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.227337 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:50.227458 28240 leader_election.cc:304] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.227552 28575 raft_consensus.cc:3060] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.227906 30852 raft_consensus.cc:2804] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.228667 28416 tablet_service.cc:1511] Processing CreateTablet for tablet a06503cd8de14a7cb726271e0b68c436 (DEFAULT_TABLE table=default.Table_d1c4892174e5421194d26b7a9cd25545 [id=d2c045a235bd4b858517a779241c8b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.229187 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a06503cd8de14a7cb726271e0b68c436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.229404 30852 raft_consensus.cc:697] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:50.229699 30852 consensus_queue.cc:237] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.231911 28736 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:50.232996 28736 ts_tablet_manager.cc:1403] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.239s user 0.004s sys 0.000s
I20260320 15:27:50.232218 30773 leader_election.cc:290] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.235355 28736 raft_consensus.cc:359] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.235905 28575 raft_consensus.cc:2468] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.239385 28736 raft_consensus.cc:385] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.240852 28736 raft_consensus.cc:740] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.241485 28736 consensus_queue.cc:260] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.242971 28736 ts_tablet_manager.cc:1434] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.009s user 0.000s sys 0.001s
I20260320 15:27:50.244222 30960 raft_consensus.cc:493] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.244385 30960 raft_consensus.cc:515] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.245128 30960 leader_election.cc:290] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:50.246412 28524 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.2:45233, user_credentials={real_user=slave}} blocked reactor thread for 63973.2us
I20260320 15:27:50.246544 29640 ts_tablet_manager.cc:1929] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.246755 29640 log.cc:1199] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/dd315b5d49104d679b03515b65ee30ac
I20260320 15:27:50.247270 29640 ts_tablet_manager.cc:1950] T dd315b5d49104d679b03515b65ee30ac P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:50.255589 28736 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:50.257364 28736 tablet_bootstrap.cc:654] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.258740 29643 ts_tablet_manager.cc:1929] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.261626 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c87a35c8fd4d87a1cd9e5ca3db15ec" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.267807 30655 raft_consensus.cc:493] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:50.267994 30655 raft_consensus.cc:515] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.268741 30655 leader_election.cc:290] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.279951 30655 raft_consensus.cc:493] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.280102 30655 raft_consensus.cc:515] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.280692 30655 leader_election.cc:290] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:50.284189 28524 consensus_peers.cc:597] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:50.285097 28559 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfcd345c77d468b8f5294e46ac18029 (DEFAULT_TABLE table=default.Table_ef3eff1b17b34682941f0ec4b03358ce [id=b7a7d1fadff747e1badfa5c83350a24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.298547 30840 ts_tablet_manager.cc:1929] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:50.298696 30840 log.cc:1199] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/9dd293a68f894cbfa122d53918816c28
I20260320 15:27:50.299122 30840 ts_tablet_manager.cc:1950] T 9dd293a68f894cbfa122d53918816c28 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:50.300127 30840 tablet_replica.cc:333] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:50.300242 28733 tablet_bootstrap.cc:492] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
W20260320 15:27:50.300302 28524 leader_election.cc:343] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:50.300345 30840 raft_consensus.cc:2243] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.300369 28733 ts_tablet_manager.cc:1403] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.183s user 0.049s sys 0.069s
I20260320 15:27:50.300524 30840 raft_consensus.cc:2272] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.301087 30840 ts_tablet_manager.cc:1916] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.301481 28736 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:50.301651 28736 ts_tablet_manager.cc:1403] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.002s
W20260320 15:27:50.301868 28524 leader_election.cc:343] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 48c1002cbdd845e2b96dcb57c450cc6d
I20260320 15:27:50.260943 28734 tablet_bootstrap.cc:654] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.302475 28736 raft_consensus.cc:359] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.302752 28736 raft_consensus.cc:385] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.302858 28736 raft_consensus.cc:740] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.303217 30395 raft_consensus.cc:493] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260320 15:27:50.265650 28382 leader_election.cc:343] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.308163 30852 raft_consensus.cc:493] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.308390 30852 raft_consensus.cc:515] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.309228 30852 leader_election.cc:290] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.310077 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.298234 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 290) took 5504 ms. Trace:
I20260320 15:27:50.303447 30395 raft_consensus.cc:515] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.310281 28444 raft_consensus.cc:2468] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:50.308892 28733 raft_consensus.cc:359] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.310381 28733 raft_consensus.cc:385] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.310248 29640 rpcz_store.cc:276] 0320 15:27:44.793683 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.793750 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:50.085408 (+5291658us) ts_tablet_manager.cc:1151] Deleting tablet dd315b5d49104d679b03515b65ee30ac
0320 15:27:50.085491 (+ 83us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.173716 (+ 88225us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.246369 (+ 72653us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.298208 (+ 51839us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5273001}
I20260320 15:27:50.310427 28733 raft_consensus.cc:740] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.310631 28733 consensus_queue.cc:260] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.310708 28240 leader_election.cc:304] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.310813 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet dd315b5d49104d679b03515b65ee30ac (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]) successfully deleted
I20260320 15:27:50.311208 28733 ts_tablet_manager.cc:1434] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260320 15:27:50.311573 28733 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.312810 30852 raft_consensus.cc:2804] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.312912 30852 raft_consensus.cc:493] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.312999 30852 raft_consensus.cc:3060] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.313079 28733 tablet_bootstrap.cc:654] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.316545 30395 leader_election.cc:290] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.316895 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.317166 28444 raft_consensus.cc:2393] T 7aba58642f7049b1aa0536b6ab6cb271 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:50.317813 30687 raft_consensus.cc:493] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.317943 30687 raft_consensus.cc:515] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.318629 30687 leader_election.cc:290] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.318939 30840 ts_tablet_manager.cc:1929] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:50.319070 30840 log.cc:1199] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/0e3f6d9a528946908891f77f951d23dd
I20260320 15:27:50.319453 30840 ts_tablet_manager.cc:1950] T 0e3f6d9a528946908891f77f951d23dd P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:50.319460 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
W20260320 15:27:50.320057 28382 leader_election.cc:343] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.269946 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 5dfcd345c77d468b8f5294e46ac18029 (DEFAULT_TABLE table=default.Table_ef3eff1b17b34682941f0ec4b03358ce [id=b7a7d1fadff747e1badfa5c83350a24e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.324373 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfcd345c77d468b8f5294e46ac18029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.326026 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 292) took 5493 ms. Trace:
I20260320 15:27:50.326150 29640 rpcz_store.cc:276] 0320 15:27:44.832514 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.832566 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:50.320539 (+5487973us) ts_tablet_manager.cc:1151] Deleting tablet 56fa691a26114657ad768545a30dfd3c
0320 15:27:50.325898 (+ 5359us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.325996 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5487772}
W20260320 15:27:50.326705 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 56fa691a26114657ad768545a30dfd3c because the tablet was not found. No further retry: Not found: Tablet not found: 56fa691a26114657ad768545a30dfd3c
I20260320 15:27:50.333139 30852 raft_consensus.cc:515] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.334983 30852 leader_election.cc:290] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.334957 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.335225 28444 raft_consensus.cc:3060] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.337616 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 293) took 5504 ms. Trace:
I20260320 15:27:50.338039 28444 raft_consensus.cc:2468] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.338552 28240 leader_election.cc:304] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.339177 30773 raft_consensus.cc:2804] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.339326 30773 raft_consensus.cc:697] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:50.339735 30773 consensus_queue.cc:237] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.337733 29640 rpcz_store.cc:276] 0320 15:27:44.832880 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.832999 (+ 119us) service_pool.cc:225] Handling call
0320 15:27:50.327204 (+5494205us) ts_tablet_manager.cc:1151] Deleting tablet 8241a342054240d2ac62b729cc9c9a62
0320 15:27:50.337480 (+ 10276us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.337586 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5494069}
W20260320 15:27:50.343509 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 8241a342054240d2ac62b729cc9c9a62 because the tablet was not found. No further retry: Not found: Tablet not found: 8241a342054240d2ac62b729cc9c9a62
I20260320 15:27:50.344483 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_ac17b8c1958d407e938050586e69cb01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:50.344707 30852 consensus_queue.cc:1048] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:50.345443 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac17b8c1958d407e938050586e69cb01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.345914 28736 consensus_queue.cc:260] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.346474 28736 ts_tablet_manager.cc:1434] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.045s user 0.001s sys 0.001s
I20260320 15:27:50.347836 28736 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:50.278697 30951 tablet.cc:1723] T 0acad11980f842e7b3a09216241e493d P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260320 15:27:50.288940 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 90149.3us
I20260320 15:27:50.353436 29643 log.cc:1199] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/5de834725bda4f249882914f1247ab62
I20260320 15:27:50.354046 29643 ts_tablet_manager.cc:1950] T 5de834725bda4f249882914f1247ab62 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:50.358942 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dfcd345c77d468b8f5294e46ac18029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.359417 30687 raft_consensus.cc:493] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.359598 30687 raft_consensus.cc:515] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.360260 30687 leader_election.cc:290] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.361186 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_b1165742e11d4e3cb02367b20289687f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260320 15:27:50.362461 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e779a274913f4caeaf82e3f180457830" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.363162 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 294) took 5530 ms. Trace:
I20260320 15:27:50.363257 29640 rpcz_store.cc:276] 0320 15:27:44.832961 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.833168 (+ 207us) service_pool.cc:225] Handling call
0320 15:27:50.347054 (+5513886us) ts_tablet_manager.cc:1151] Deleting tablet 889d620c311a42039c22af53ad0802f2
0320 15:27:50.361942 (+ 14888us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.363147 (+ 1205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5513746}
W20260320 15:27:50.363824 28382 leader_election.cc:343] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:50.363821 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 889d620c311a42039c22af53ad0802f2 because the tablet was not found. No further retry: Not found: Tablet not found: 889d620c311a42039c22af53ad0802f2
W20260320 15:27:50.364130 28239 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.3:44463, user_credentials={real_user=slave}} blocked reactor thread for 136101us
I20260320 15:27:50.364221 30852 consensus_queue.cc:1048] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.364820 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26ae379304674bc3941af125f0d9d6b3" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.365015 28574 raft_consensus.cc:2376] T 26ae379304674bc3941af125f0d9d6b3 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1. Re-sending same reply.
I20260320 15:27:50.365286 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
I20260320 15:27:50.365525 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 441) took 1413 ms. Trace:
I20260320 15:27:50.365644 29643 rpcz_store.cc:276] 0320 15:27:48.952107 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:48.954947 (+ 2840us) service_pool.cc:225] Handling call
0320 15:27:50.013732 (+1058785us) ts_tablet_manager.cc:1151] Deleting tablet 5de834725bda4f249882914f1247ab62
0320 15:27:50.013744 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.255189 (+241445us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.258613 (+ 3424us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.365499 (+106886us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1058538}
I20260320 15:27:50.366001 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]) successfully deleted
I20260320 15:27:50.366091 29643 tablet_replica.cc:333] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:50.366246 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 5de834725bda4f249882914f1247ab62 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]) successfully deleted
I20260320 15:27:50.366339 29643 raft_consensus.cc:2243] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.366501 29643 raft_consensus.cc:2272] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.367187 29643 ts_tablet_manager.cc:1916] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.367395 28304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09c87a35c8fd4d87a1cd9e5ca3db15ec" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.367657 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.368013 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 297) took 5534 ms. Trace:
I20260320 15:27:50.368142 29640 rpcz_store.cc:276] 0320 15:27:44.833979 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.834019 (+ 40us) service_pool.cc:225] Handling call
0320 15:27:50.367882 (+5533863us) ts_tablet_manager.cc:1151] Deleting tablet 45a0e8e2ce7a454ebcab4e59a4de68dd
0320 15:27:50.367895 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.367984 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5533698}
I20260320 15:27:50.368578 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 296) took 5534 ms. Trace:
I20260320 15:27:50.371426 28576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
W20260320 15:27:50.371694 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 45a0e8e2ce7a454ebcab4e59a4de68dd because the tablet was not found. No further retry: Not found: Tablet not found: 45a0e8e2ce7a454ebcab4e59a4de68dd
W20260320 15:27:50.372490 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1165742e11d4e3cb02367b20289687f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.372678 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.368670 29640 rpcz_store.cc:276] 0320 15:27:44.833879 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.834332 (+ 453us) service_pool.cc:225] Handling call
0320 15:27:50.368482 (+5534150us) ts_tablet_manager.cc:1151] Deleting tablet b03e7555b9a646b599acb0c7dff90c9a
0320 15:27:50.368494 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.368560 (+ 66us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5533999}
I20260320 15:27:50.374959 30655 raft_consensus.cc:493] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.375196 30655 raft_consensus.cc:515] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.375854 30655 leader_election.cc:290] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.376648 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 299) took 5541 ms. Trace:
I20260320 15:27:50.376596 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.376776 29640 rpcz_store.cc:276] 0320 15:27:44.834837 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.834883 (+ 46us) service_pool.cc:225] Handling call
0320 15:27:50.376548 (+5541665us) ts_tablet_manager.cc:1151] Deleting tablet ecaa08ba34514c5e99a9e2d4ba9268a9
0320 15:27:50.376561 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.376632 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5541514}
W20260320 15:27:50.376899 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet b03e7555b9a646b599acb0c7dff90c9a because the tablet was not found. No further retry: Not found: Tablet not found: b03e7555b9a646b599acb0c7dff90c9a
I20260320 15:27:50.377215 29640 tablet_replica.cc:333] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
W20260320 15:27:50.377197 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ecaa08ba34514c5e99a9e2d4ba9268a9 because the tablet was not found. No further retry: Not found: Tablet not found: ecaa08ba34514c5e99a9e2d4ba9268a9
I20260320 15:27:50.377430 29640 raft_consensus.cc:2243] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:50.377807 29640 raft_consensus.cc:2272] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.378433 29640 ts_tablet_manager.cc:1916] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.390672 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:50.391278 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:50.392695 28904 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33394:
name: "default.Table_0ec651f9c0604b82a569f618b6f06b79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260320 15:27:50.393435 28904 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ec651f9c0604b82a569f618b6f06b79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260320 15:27:50.396384 28736 tablet_bootstrap.cc:654] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.402998 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e779a274913f4caeaf82e3f180457830" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.408166 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:50.408416 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:50.408675 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ab9893da469449bba556d11bd347d89b (table default.Table_f57610d7b6234bb9bcc7aa518b2bca0f [id=910ef5cd98e1495f829465985895c17b]). Expected version 1 got 0
I20260320 15:27:50.408859 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:50.409040 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:50.409466 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7aba58642f7049b1aa0536b6ab6cb271" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.409597 28901 catalog_manager.cc:5671] T bcf4e017e2e743d7b38628d6f1e0d72e P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2). New cstate: current_term: 1 leader_uuid: "d22e73471b9046d28d3cf63c905b11b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:50.410276 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:50.410712 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (table default.Table_7ac4015d018a4c8e96ea949d122786a3 [id=7be232d8d79846eb858e6dd529427db4]). Expected version 1 got 0
I20260320 15:27:50.411044 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:50.411211 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:50.412066 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet f59a3c9fa17a4babaa9a18f18d51f193 from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:50.418886 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb011d672f244585b90272443897ed5a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
W20260320 15:27:50.419170 28239 leader_election.cc:343] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:50.420107 28239 leader_election.cc:343] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.420454 30466 consensus_queue.cc:1048] T 233f6a8e9369411b8e0c35a167c55ff6 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:50.421747 28383 leader_election.cc:343] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.421873 28383 leader_election.cc:304] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.422266 30687 raft_consensus.cc:2749] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:50.422819 28383 leader_election.cc:343] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.422942 28383 leader_election.cc:304] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
W20260320 15:27:50.423134 28862 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 55bd68c6e4364a038a815d1fc05ad25e (table default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55bd68c6e4364a038a815d1fc05ad25e consensus state has no leader
I20260320 15:27:50.423347 30687 raft_consensus.cc:2749] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:50.426611 28873 catalog_manager.cc:5075] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): alter failed for tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]),no further retry: Not found: Tablet not found: 1ccfd2d5e82448b99f1022954b98c6cc
W20260320 15:27:50.428014 28383 leader_election.cc:343] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.428160 28383 leader_election.cc:304] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.428623 30960 raft_consensus.cc:2749] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.428800 30949 tablet.cc:1723] T f8f70142c3a449439a0a475bddafdd40 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260320 15:27:50.430385 28524 consensus_peers.cc:597] T 5de834725bda4f249882914f1247ab62 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5de834725bda4f249882914f1247ab62. This is attempt 1: this message will repeat every 5th retry.
W20260320 15:27:50.430857 28524 leader_election.cc:343] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:50.431013 28524 leader_election.cc:304] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.431713 30395 raft_consensus.cc:2749] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:50.431777 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:50.432188 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:50.433207 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:50.433583 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:50.433780 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:50.434080 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 4b527973ede040e286f7b8f2e7536699 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:50.434445 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:50.434648 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:50.434911 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:50.435199 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:50.436663 29640 ts_tablet_manager.cc:1929] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.436787 29640 log.cc:1199] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/41da3cb06b1147fbae2695bc5c9ffaa5
I20260320 15:27:50.437112 29640 ts_tablet_manager.cc:1950] T 41da3cb06b1147fbae2695bc5c9ffaa5 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
W20260320 15:27:50.439078 28524 leader_election.cc:343] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 48c1002cbdd845e2b96dcb57c450cc6d
I20260320 15:27:50.439198 28524 leader_election.cc:304] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.439728 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 291) took 5607 ms. Trace:
I20260320 15:27:50.439862 30395 raft_consensus.cc:2749] T 48c1002cbdd845e2b96dcb57c450cc6d P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.442262 28733 tablet_bootstrap.cc:492] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.442405 28733 ts_tablet_manager.cc:1403] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.131s user 0.008s sys 0.028s
I20260320 15:27:50.443102 28733 raft_consensus.cc:359] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.443367 28733 raft_consensus.cc:385] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.443446 28733 raft_consensus.cc:740] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.443691 28733 consensus_queue.cc:260] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.444566 28733 ts_tablet_manager.cc:1434] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:50.444880 28733 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.446259 30773 raft_consensus.cc:493] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.446445 28733 tablet_bootstrap.cc:654] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.447255 30773 raft_consensus.cc:515] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.448969 30773 leader_election.cc:290] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.449478 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.449812 28447 raft_consensus.cc:2468] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:50.450322 30951 tablet.cc:1723] T f8f70142c3a449439a0a475bddafdd40 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.450970 28240 leader_election.cc:304] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.453776 30773 raft_consensus.cc:2804] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.455279 30773 raft_consensus.cc:493] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.455865 28524 leader_election.cc:304] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.455947 30773 raft_consensus.cc:3060] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.439829 29640 rpcz_store.cc:276] 0320 15:27:44.832399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.835264 (+ 2865us) service_pool.cc:225] Handling call
0320 15:27:50.377161 (+5541897us) ts_tablet_manager.cc:1151] Deleting tablet 41da3cb06b1147fbae2695bc5c9ffaa5
0320 15:27:50.377172 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.402098 (+ 24926us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.436546 (+ 34448us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.439711 (+ 3165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5541746}
I20260320 15:27:50.456693 30395 raft_consensus.cc:3060] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.457015 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 41da3cb06b1147fbae2695bc5c9ffaa5 (table default.Table_e4276cc8c36a438b8f87af916b9fe8e3 [id=8838ed5b7ba7458f940e220728d87565]) successfully deleted
W20260320 15:27:50.457536 28524 consensus_peers.cc:597] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5095b42b69d40539118c1f63a95f795. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:50.458441 28524 leader_election.cc:304] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.458673 30655 raft_consensus.cc:2749] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.458685 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 300) took 5623 ms. Trace:
I20260320 15:27:50.459302 30395 raft_consensus.cc:2749] T 7aba58642f7049b1aa0536b6ab6cb271 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.459331 29640 rpcz_store.cc:276] 0320 15:27:44.835218 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.835473 (+ 255us) service_pool.cc:225] Handling call
0320 15:27:50.458584 (+5623111us) ts_tablet_manager.cc:1151] Deleting tablet ffe278739bbc4ddab848af9c25241380
0320 15:27:50.458593 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.458667 (+ 74us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5622964}
W20260320 15:27:50.459895 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ffe278739bbc4ddab848af9c25241380 because the tablet was not found. No further retry: Not found: Tablet not found: ffe278739bbc4ddab848af9c25241380
I20260320 15:27:50.460091 30773 raft_consensus.cc:515] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.460390 29640 tablet_replica.cc:333] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:50.460601 29640 raft_consensus.cc:2243] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.460772 29640 raft_consensus.cc:2272] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.466245 30949 tablet.cc:1723] T 3c7a605bdcf14d2ab64ad078efd3d999 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.466150 30986 tablet.cc:1723] T 3c7a605bdcf14d2ab64ad078efd3d999 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.469307 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.469857 28437 raft_consensus.cc:3060] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.470368 29640 ts_tablet_manager.cc:1916] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.471166 28901 catalog_manager.cc:5671] T 7dfbc7338eda489987315e681ee44146 P d22e73471b9046d28d3cf63c905b11b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } }
I20260320 15:27:50.472314 28736 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:50.472472 28736 ts_tablet_manager.cc:1403] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.125s user 0.003s sys 0.000s
I20260320 15:27:50.473165 28736 raft_consensus.cc:359] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.473351 28736 raft_consensus.cc:385] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.473436 28736 raft_consensus.cc:740] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.473564 28437 raft_consensus.cc:2468] T afb692aab24849bc8e50086927421e13 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.473704 28736 consensus_queue.cc:260] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.474053 28240 leader_election.cc:304] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.475605 30971 raft_consensus.cc:2804] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.475739 30971 raft_consensus.cc:697] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:50.477031 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:50.477203 30971 consensus_queue.cc:237] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.476191 30773 leader_election.cc:290] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.479151 28736 ts_tablet_manager.cc:1434] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260320 15:27:50.480520 28734 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.480759 28734 ts_tablet_manager.cc:1403] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.266s user 0.023s sys 0.023s
I20260320 15:27:50.482893 28734 raft_consensus.cc:359] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.483098 28734 raft_consensus.cc:385] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.483156 28734 raft_consensus.cc:740] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.483398 28734 consensus_queue.cc:260] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.485478 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:50.485544 28734 ts_tablet_manager.cc:1434] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260320 15:27:50.485680 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 0acad11980f842e7b3a09216241e493d (table default.Table_fdf1117756d04248ad30f6a0c9101c6f [id=e703fc2e6e2345138d5a6852097ade20]). Expected version 1 got 0
I20260320 15:27:50.486181 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:50.487237 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:50.487298 28734 tablet_bootstrap.cc:492] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.487567 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:50.488667 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:50.489025 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:50.489473 28901 catalog_manager.cc:5763] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:50.499282 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.501752 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb692aab24849bc8e50086927421e13" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:50.510275 28733 tablet_bootstrap.cc:492] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.510455 28733 ts_tablet_manager.cc:1403] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.013s
I20260320 15:27:50.511173 28733 raft_consensus.cc:359] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.511371 28733 raft_consensus.cc:385] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.511427 28733 raft_consensus.cc:740] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.511726 28733 consensus_queue.cc:260] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.512441 28733 ts_tablet_manager.cc:1434] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:50.512959 28733 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.514446 28733 tablet_bootstrap.cc:654] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.516222 28736 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:50.502720 30852 raft_consensus.cc:493] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.516558 30852 raft_consensus.cc:515] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.517704 30852 leader_election.cc:290] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.512768 30971 consensus_queue.cc:1048] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.499998 30773 consensus_queue.cc:1048] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.523278 29640 ts_tablet_manager.cc:1929] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.523430 29640 log.cc:1199] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/6d9b0f1936f54c699211c5ee9709f9ed
I20260320 15:27:50.523823 29640 ts_tablet_manager.cc:1950] T 6d9b0f1936f54c699211c5ee9709f9ed P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:50.524175 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 301) took 5688 ms. Trace:
I20260320 15:27:50.524130 28437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.524267 29640 rpcz_store.cc:276] 0320 15:27:44.836029 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.836081 (+ 52us) service_pool.cc:225] Handling call
0320 15:27:50.460330 (+5624249us) ts_tablet_manager.cc:1151] Deleting tablet 6d9b0f1936f54c699211c5ee9709f9ed
0320 15:27:50.460340 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.493442 (+ 33102us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.523125 (+ 29683us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.524158 (+ 1033us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5624096}
I20260320 15:27:50.524375 28437 raft_consensus.cc:2468] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:50.524811 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 6d9b0f1936f54c699211c5ee9709f9ed (table default.Table_8ad5a03705764f249c6964341f957801 [id=0302c040a4c440f59a0402eab94a8844]) successfully deleted
I20260320 15:27:50.526407 28736 tablet_bootstrap.cc:654] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.528842 28733 tablet_bootstrap.cc:492] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.528996 28733 ts_tablet_manager.cc:1403] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.005s
I20260320 15:27:50.529670 28733 raft_consensus.cc:359] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.529845 28733 raft_consensus.cc:385] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.529891 28733 raft_consensus.cc:740] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.530018 28350 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:50.530119 28733 consensus_queue.cc:260] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.531792 28733 ts_tablet_manager.cc:1434] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:50.533762 28733 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.535604 28733 tablet_bootstrap.cc:654] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.536108 28240 leader_election.cc:304] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.537107 30971 raft_consensus.cc:2804] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.537256 30971 raft_consensus.cc:493] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.537397 30971 raft_consensus.cc:3060] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.543431 28734 tablet_bootstrap.cc:654] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.540794 30971 raft_consensus.cc:515] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.552739 30971 leader_election.cc:290] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.553663 28439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.553911 28439 raft_consensus.cc:3060] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.557840 28439 raft_consensus.cc:2468] T 50c1398534d84701ae7085a037f236ad P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.561748 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet f59a3c9fa17a4babaa9a18f18d51f193
I20260320 15:27:50.562319 28904 catalog_manager.cc:6741] ReplaceTablet: tablet f59a3c9fa17a4babaa9a18f18d51f193 deleted and replaced by tablet 3285b579325e4ed0ba2f97a1a2e2c4d0
I20260320 15:27:50.562582 28904 master_service.cc:954] ReplaceTablet: received request to replace tablet dd315b5d49104d679b03515b65ee30ac from {username='slave'} at 127.0.0.1:33394
I20260320 15:27:50.563304 28904 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_dfac030930b54821b08b28ee38374406" } modify_external_catalogs: true
I20260320 15:27:50.563607 28904 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33394:
table { table_name: "default.Table_dfac030930b54821b08b28ee38374406" } modify_external_catalogs: true
I20260320 15:27:50.564332 30967 raft_consensus.cc:493] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.564515 30967 raft_consensus.cc:515] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.565140 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 with delete_type TABLET_DATA_DELETED (replaced by tablet 3285b579325e4ed0ba2f97a1a2e2c4d0) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:50.565317 30967 leader_election.cc:290] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.565968 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d12bd6ae67f4600a0d54e7d5d502dda" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.566445 28425 tablet_service.cc:1558] Processing DeleteTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 with delete_type TABLET_DATA_DELETED (replaced by tablet 3285b579325e4ed0ba2f97a1a2e2c4d0) from {username='slave'} at 127.0.0.1:55030
W20260320 15:27:50.566593 28382 leader_election.cc:343] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.559892 28240 leader_election.cc:304] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.563162 30952 tablet.cc:1723] T f8f70142c3a449439a0a475bddafdd40 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.569478 30971 raft_consensus.cc:2804] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.574501 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 302) took 5738 ms. Trace:
I20260320 15:27:50.574690 29640 rpcz_store.cc:276] 0320 15:27:44.836367 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.836408 (+ 41us) service_pool.cc:225] Handling call
0320 15:27:50.574329 (+5737921us) ts_tablet_manager.cc:1151] Deleting tablet 1bf5e17dea5445d9909a229fa8315348
0320 15:27:50.574348 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.574464 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5737730}
W20260320 15:27:50.575485 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 1bf5e17dea5445d9909a229fa8315348 because the tablet was not found. No further retry: Not found: Tablet not found: 1bf5e17dea5445d9909a229fa8315348
I20260320 15:27:50.576188 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 303) took 5739 ms. Trace:
I20260320 15:27:50.576392 29640 rpcz_store.cc:276] 0320 15:27:44.836715 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.836820 (+ 105us) service_pool.cc:225] Handling call
0320 15:27:50.576049 (+5739229us) ts_tablet_manager.cc:1151] Deleting tablet c3bc4dd47817490d88725127b456cc70
0320 15:27:50.576067 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.576162 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5739037}
W20260320 15:27:50.577235 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet c3bc4dd47817490d88725127b456cc70 because the tablet was not found. No further retry: Not found: Tablet not found: c3bc4dd47817490d88725127b456cc70
I20260320 15:27:50.577693 30971 raft_consensus.cc:697] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:50.577870 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 304) took 5718 ms. Trace:
I20260320 15:27:50.578011 30971 consensus_queue.cc:237] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.578051 29640 rpcz_store.cc:276] 0320 15:27:44.859503 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.859588 (+ 85us) service_pool.cc:225] Handling call
0320 15:27:50.577701 (+5718113us) ts_tablet_manager.cc:1151] Deleting tablet ea6ff59d2ffe4d579e6db2375eb11414
0320 15:27:50.577711 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.577846 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5717879}
W20260320 15:27:50.578806 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ea6ff59d2ffe4d579e6db2375eb11414 because the tablet was not found. No further retry: Not found: Tablet not found: ea6ff59d2ffe4d579e6db2375eb11414
I20260320 15:27:50.579206 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 305) took 5717 ms. Trace:
I20260320 15:27:50.584352 30949 tablet.cc:1723] T 325caab9b8f34f04a97b41137652ccdb P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.590355 28734 tablet_bootstrap.cc:492] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.590691 28734 ts_tablet_manager.cc:1403] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.103s user 0.003s sys 0.001s
I20260320 15:27:50.591517 28734 raft_consensus.cc:359] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.591902 28734 raft_consensus.cc:385] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.592013 28734 raft_consensus.cc:740] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.592303 28734 consensus_queue.cc:260] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.593088 30655 raft_consensus.cc:493] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.593294 30655 raft_consensus.cc:515] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
W20260320 15:27:50.563768 28239 leader_election.cc:343] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.594409 30655 leader_election.cc:290] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.595242 28734 ts_tablet_manager.cc:1434] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
W20260320 15:27:50.595432 28524 leader_election.cc:343] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 49654c67435a48448794879464f6cfb2
W20260320 15:27:50.595868 28239 leader_election.cc:343] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.605852 28736 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:50.606012 28736 ts_tablet_manager.cc:1403] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.090s user 0.001s sys 0.001s
I20260320 15:27:50.606685 28736 raft_consensus.cc:359] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.606927 28736 raft_consensus.cc:385] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.606982 28736 raft_consensus.cc:740] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.607172 28736 consensus_queue.cc:260] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.609054 28736 ts_tablet_manager.cc:1434] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:50.610325 28736 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:50.610644 29640 rpcz_store.cc:276] 0320 15:27:44.861702 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.862347 (+ 645us) service_pool.cc:225] Handling call
0320 15:27:50.579111 (+5716764us) ts_tablet_manager.cc:1151] Deleting tablet 5fa5f6fe801b4452bf294a9ae23bb163
0320 15:27:50.579121 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.579186 (+ 65us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5716555}
I20260320 15:27:50.566964 30466 consensus_queue.cc:1048] T d87a2f5513684da79f6eebbed89d0392 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.614174 28734 tablet_bootstrap.cc:492] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:50.615139 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 5fa5f6fe801b4452bf294a9ae23bb163 because the tablet was not found. No further retry: Not found: Tablet not found: 5fa5f6fe801b4452bf294a9ae23bb163
I20260320 15:27:50.615842 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 306) took 5736 ms. Trace:
I20260320 15:27:50.611799 30655 consensus_queue.cc:1048] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.618228 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.619652 30951 tablet.cc:1723] T 325caab9b8f34f04a97b41137652ccdb P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260320 15:27:50.620774 28383 consensus_peers.cc:597] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9dd293a68f894cbfa122d53918816c28. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:50.621292 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50c1398534d84701ae7085a037f236ad" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:50.621631 28733 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.621767 28733 ts_tablet_manager.cc:1403] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.088s user 0.008s sys 0.028s
I20260320 15:27:50.622539 28733 raft_consensus.cc:359] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.622738 28733 raft_consensus.cc:385] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.622792 28733 raft_consensus.cc:740] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.623009 28733 consensus_queue.cc:260] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.626580 29640 rpcz_store.cc:276] 0320 15:27:44.878987 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.879043 (+ 56us) service_pool.cc:225] Handling call
0320 15:27:50.615708 (+5736665us) ts_tablet_manager.cc:1151] Deleting tablet 2ee4a0d559ce4dfea878bd3f1c03c09a
0320 15:27:50.615725 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.615821 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5736414}
W20260320 15:27:50.631215 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 2ee4a0d559ce4dfea878bd3f1c03c09a because the tablet was not found. No further retry: Not found: Tablet not found: 2ee4a0d559ce4dfea878bd3f1c03c09a
I20260320 15:27:50.632984 28734 tablet_bootstrap.cc:654] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.636974 28268 tablet_service.cc:1558] Processing DeleteTablet for tablet f59a3c9fa17a4babaa9a18f18d51f193 with delete_type TABLET_DATA_DELETED (replaced by tablet 3285b579325e4ed0ba2f97a1a2e2c4d0) from {username='slave'} at 127.0.0.1:46760
I20260320 15:27:50.637837 30840 tablet_replica.cc:333] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: stopping tablet replica
I20260320 15:27:50.638067 30840 raft_consensus.cc:2243] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:50.638079 28294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d12bd6ae67f4600a0d54e7d5d502dda" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.638387 30840 raft_consensus.cc:2272] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.639034 30840 ts_tablet_manager.cc:1916] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.641875 28736 tablet_bootstrap.cc:654] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.644017 30952 tablet.cc:1723] T 3c7a605bdcf14d2ab64ad078efd3d999 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.644351 28733 ts_tablet_manager.cc:1434] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20260320 15:27:50.644716 28733 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.647879 28733 tablet_bootstrap.cc:654] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.651896 30840 ts_tablet_manager.cc:1929] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:50.652124 30840 log.cc:1199] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-0/wal/wals/f59a3c9fa17a4babaa9a18f18d51f193
I20260320 15:27:50.652595 30840 ts_tablet_manager.cc:1950] T f59a3c9fa17a4babaa9a18f18d51f193 P 3ce5383d28b347ab979765b880e5773c: Deleting consensus metadata
I20260320 15:27:50.654047 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 309) took 5747 ms. Trace:
I20260320 15:27:50.659587 29640 rpcz_store.cc:276] 0320 15:27:44.906528 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.907224 (+ 696us) service_pool.cc:225] Handling call
0320 15:27:50.653917 (+5746693us) ts_tablet_manager.cc:1151] Deleting tablet 7feb8477ebf841a19977a434f365f537
0320 15:27:50.653930 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.654025 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5746514}
I20260320 15:27:50.659811 28733 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.660066 28733 ts_tablet_manager.cc:1403] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
W20260320 15:27:50.660414 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7feb8477ebf841a19977a434f365f537 because the tablet was not found. No further retry: Not found: Tablet not found: 7feb8477ebf841a19977a434f365f537
I20260320 15:27:50.660871 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 311) took 5728 ms. Trace:
I20260320 15:27:50.660959 28733 raft_consensus.cc:359] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.661127 28733 raft_consensus.cc:385] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.661186 28733 raft_consensus.cc:740] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.661001 29640 rpcz_store.cc:276] 0320 15:27:44.932435 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.932496 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:50.660769 (+5728273us) ts_tablet_manager.cc:1151] Deleting tablet 9176e734212849aab8d99473de02cc79
0320 15:27:50.660778 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.660854 (+ 76us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5728033}
I20260320 15:27:50.661495 28733 consensus_queue.cc:260] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:50.661710 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 9176e734212849aab8d99473de02cc79 because the tablet was not found. No further retry: Not found: Tablet not found: 9176e734212849aab8d99473de02cc79
I20260320 15:27:50.662257 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 312) took 5728 ms. Trace:
I20260320 15:27:50.662719 28733 ts_tablet_manager.cc:1434] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:50.666700 28733 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.666388 29640 rpcz_store.cc:276] 0320 15:27:44.933407 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:44.933460 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:50.662158 (+5728698us) ts_tablet_manager.cc:1151] Deleting tablet 7be03035dc3d44f9aa5b43b49281a940
0320 15:27:50.662167 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.662240 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5728527}
I20260320 15:27:50.633612 28493 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
W20260320 15:27:50.666350 28383 leader_election.cc:343] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.667507 28383 leader_election.cc:304] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.668370 30960 raft_consensus.cc:2749] T 4d12bd6ae67f4600a0d54e7d5d502dda P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:50.668473 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 7be03035dc3d44f9aa5b43b49281a940 because the tablet was not found. No further retry: Not found: Tablet not found: 7be03035dc3d44f9aa5b43b49281a940
I20260320 15:27:50.668972 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 313) took 5625 ms. Trace:
I20260320 15:27:50.669062 29640 rpcz_store.cc:276] 0320 15:27:45.043428 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.043495 (+ 67us) service_pool.cc:225] Handling call
0320 15:27:50.668857 (+5625362us) ts_tablet_manager.cc:1151] Deleting tablet ef09e4afdd1f4fce9caec4dcafa2c50c
0320 15:27:50.668869 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.668952 (+ 83us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5625069}
W20260320 15:27:50.669633 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet ef09e4afdd1f4fce9caec4dcafa2c50c because the tablet was not found. No further retry: Not found: Tablet not found: ef09e4afdd1f4fce9caec4dcafa2c50c
W20260320 15:27:50.669961 28524 leader_election.cc:343] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 49654c67435a48448794879464f6cfb2
I20260320 15:27:50.670097 28524 leader_election.cc:304] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:50.670591 30629 raft_consensus.cc:2749] T 49654c67435a48448794879464f6cfb2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:50.671406 28733 tablet_bootstrap.cc:654] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.671976 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 314) took 5627 ms. Trace:
I20260320 15:27:50.672132 29640 rpcz_store.cc:276] 0320 15:27:45.044409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.044462 (+ 53us) service_pool.cc:225] Handling call
0320 15:27:50.671876 (+5627414us) ts_tablet_manager.cc:1151] Deleting tablet 533105ad6e054fe3a4db32855bc688bd
0320 15:27:50.671887 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.671958 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5627196}
I20260320 15:27:50.672416 28873 catalog_manager.cc:5002] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]) successfully deleted
W20260320 15:27:50.672822 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 533105ad6e054fe3a4db32855bc688bd because the tablet was not found. No further retry: Not found: Tablet not found: 533105ad6e054fe3a4db32855bc688bd
W20260320 15:27:50.672948 28239 leader_election.cc:343] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.673273 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 317) took 5626 ms. Trace:
W20260320 15:27:50.673331 28239 leader_election.cc:343] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.673393 29640 rpcz_store.cc:276] 0320 15:27:45.047125 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.047373 (+ 248us) service_pool.cc:225] Handling call
0320 15:27:50.673181 (+5625808us) ts_tablet_manager.cc:1151] Deleting tablet bdd8f5ccbd35418684fd27459c7a6b9f
0320 15:27:50.673192 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.673256 (+ 64us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5625550}
I20260320 15:27:50.673661 30852 consensus_queue.cc:1048] T 7aba58642f7049b1aa0536b6ab6cb271 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260320 15:27:50.673954 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet bdd8f5ccbd35418684fd27459c7a6b9f because the tablet was not found. No further retry: Not found: Tablet not found: bdd8f5ccbd35418684fd27459c7a6b9f
I20260320 15:27:50.674384 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 318) took 5626 ms. Trace:
I20260320 15:27:50.674592 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 014d465f845e4762ba845034b2c649a6 (DEFAULT_TABLE table=default.Table_e9e7782bae064c90a600dabc73647497 [id=002755864a8f4683ace4c9d0c2a2cedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.674542 29640 rpcz_store.cc:276] 0320 15:27:45.047626 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.047668 (+ 42us) service_pool.cc:225] Handling call
0320 15:27:50.674284 (+5626616us) ts_tablet_manager.cc:1151] Deleting tablet 20a7164db1e0447fb424d8e46b747f52
0320 15:27:50.674294 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.674364 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5626466}
I20260320 15:27:50.675019 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d465f845e4762ba845034b2c649a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.675733 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 20a7164db1e0447fb424d8e46b747f52 because the tablet was not found. No further retry: Not found: Tablet not found: 20a7164db1e0447fb424d8e46b747f52
I20260320 15:27:50.676144 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 319) took 5628 ms. Trace:
I20260320 15:27:50.676270 29640 rpcz_store.cc:276] 0320 15:27:45.048061 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.048509 (+ 448us) service_pool.cc:225] Handling call
0320 15:27:50.676052 (+5627543us) ts_tablet_manager.cc:1151] Deleting tablet 07af28f257d745a1bb4173aad09045d9
0320 15:27:50.676063 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.676128 (+ 65us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5627365}
W20260320 15:27:50.676823 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 07af28f257d745a1bb4173aad09045d9 because the tablet was not found. No further retry: Not found: Tablet not found: 07af28f257d745a1bb4173aad09045d9
I20260320 15:27:50.677196 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 316) took 5630 ms. Trace:
I20260320 15:27:50.677313 29640 rpcz_store.cc:276] 0320 15:27:45.046587 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.049869 (+ 3282us) service_pool.cc:225] Handling call
0320 15:27:50.677113 (+5627244us) ts_tablet_manager.cc:1151] Deleting tablet fc009401bb3d440f9d36e080efef1204
0320 15:27:50.677122 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.677179 (+ 57us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5626954}
W20260320 15:27:50.677811 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet fc009401bb3d440f9d36e080efef1204 because the tablet was not found. No further retry: Not found: Tablet not found: fc009401bb3d440f9d36e080efef1204
I20260320 15:27:50.678188 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 320) took 5579 ms. Trace:
I20260320 15:27:50.680365 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 014d465f845e4762ba845034b2c649a6 (DEFAULT_TABLE table=default.Table_e9e7782bae064c90a600dabc73647497 [id=002755864a8f4683ace4c9d0c2a2cedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.680910 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d465f845e4762ba845034b2c649a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.687004 28416 tablet_service.cc:1511] Processing CreateTablet for tablet ec195522725942348e84236ac907be44 (DEFAULT_TABLE table=default.Table_4afe8b5be24444de80274a2c43ad204b [id=4a1edae7107844d5973fb94b141bb122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.687592 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec195522725942348e84236ac907be44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.704561 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 1d1139da116240ff8b086f888905f953 (DEFAULT_TABLE table=default.Table_fa5403c78a5f4823959f74237f673da6 [id=951f820a52c44a3fad1675f2cc1be417]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.705180 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d1139da116240ff8b086f888905f953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.709636 30960 raft_consensus.cc:493] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.709779 30960 raft_consensus.cc:515] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.710436 30960 leader_election.cc:290] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.695158 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 014d465f845e4762ba845034b2c649a6 (DEFAULT_TABLE table=default.Table_e9e7782bae064c90a600dabc73647497 [id=002755864a8f4683ace4c9d0c2a2cedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.716189 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 014d465f845e4762ba845034b2c649a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.702296 28559 tablet_service.cc:1511] Processing CreateTablet for tablet ec195522725942348e84236ac907be44 (DEFAULT_TABLE table=default.Table_4afe8b5be24444de80274a2c43ad204b [id=4a1edae7107844d5973fb94b141bb122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.704551 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 1d1139da116240ff8b086f888905f953 (DEFAULT_TABLE table=default.Table_fa5403c78a5f4823959f74237f673da6 [id=951f820a52c44a3fad1675f2cc1be417]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.715157 29640 rpcz_store.cc:276] 0320 15:27:45.098523 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.099144 (+ 621us) service_pool.cc:225] Handling call
0320 15:27:50.678083 (+5578939us) ts_tablet_manager.cc:1151] Deleting tablet a185f84c24ea402684aa1cc3628d4f11
0320 15:27:50.678091 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.678172 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5578758}
I20260320 15:27:50.720130 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d1139da116240ff8b086f888905f953. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.720286 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet a185f84c24ea402684aa1cc3628d4f11 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a185f84c24ea402684aa1cc3628d4f11 already in progress: creating tablet
I20260320 15:27:50.721791 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 321) took 5619 ms. Trace:
I20260320 15:27:50.711001 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ea5e924a734c8ba6dfb641359593a3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.714036 30655 raft_consensus.cc:493] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.723277 30655 raft_consensus.cc:515] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.708464 30629 raft_consensus.cc:493] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:50.723692 30629 raft_consensus.cc:515] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.724465 30629 leader_election.cc:290] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233)
I20260320 15:27:50.725883 30655 leader_election.cc:290] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
W20260320 15:27:50.726416 28382 leader_election.cc:343] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.727083 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.727281 28444 raft_consensus.cc:2468] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:50.727690 28524 leader_election.cc:304] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.727938 30655 raft_consensus.cc:2804] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260320 15:27:50.727989 28524 leader_election.cc:343] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:50.728034 30655 raft_consensus.cc:493] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.728104 30655 raft_consensus.cc:3060] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.730695 30655 raft_consensus.cc:515] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.731388 30655 leader_election.cc:290] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.732709 28734 tablet_bootstrap.cc:492] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.732939 28559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec195522725942348e84236ac907be44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.732978 28734 ts_tablet_manager.cc:1403] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.119s user 0.004s sys 0.000s
I20260320 15:27:50.733981 28734 raft_consensus.cc:359] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.734253 28734 raft_consensus.cc:385] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.734334 28734 raft_consensus.cc:740] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.734648 28734 consensus_queue.cc:260] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.735805 28734 ts_tablet_manager.cc:1434] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:50.721912 29640 rpcz_store.cc:276] 0320 15:27:45.102109 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.102175 (+ 66us) service_pool.cc:225] Handling call
0320 15:27:50.721631 (+5619456us) ts_tablet_manager.cc:1151] Deleting tablet 0e9a4e61bab44d768d9adfed00c945e9
0320 15:27:50.721648 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.721762 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5619234}
W20260320 15:27:50.736678 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 0e9a4e61bab44d768d9adfed00c945e9 because the tablet was not found. No further retry: Not found: Tablet not found: 0e9a4e61bab44d768d9adfed00c945e9
I20260320 15:27:50.737190 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 322) took 5634 ms. Trace:
I20260320 15:27:50.737320 29640 rpcz_store.cc:276] 0320 15:27:45.102912 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.102970 (+ 58us) service_pool.cc:225] Handling call
0320 15:27:50.737052 (+5634082us) ts_tablet_manager.cc:1151] Deleting tablet e7bd0616f5604f8fbb8edd8e9c230f39
0320 15:27:50.737067 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.737165 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5633890}
W20260320 15:27:50.737890 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet e7bd0616f5604f8fbb8edd8e9c230f39 because the tablet was not found. No further retry: Not found: Tablet not found: e7bd0616f5604f8fbb8edd8e9c230f39
I20260320 15:27:50.738201 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 324) took 5631 ms. Trace:
I20260320 15:27:50.738286 29640 rpcz_store.cc:276] 0320 15:27:45.106599 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.106647 (+ 48us) service_pool.cc:225] Handling call
0320 15:27:50.738113 (+5631466us) ts_tablet_manager.cc:1151] Deleting tablet 43a0fadc7dac4e21ad034e4cd2d70826
0320 15:27:50.738123 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.738186 (+ 63us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5631273}
W20260320 15:27:50.738664 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 43a0fadc7dac4e21ad034e4cd2d70826 because the tablet was not found. No further retry: Not found: Tablet not found: 43a0fadc7dac4e21ad034e4cd2d70826
I20260320 15:27:50.738960 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 323) took 5632 ms. Trace:
I20260320 15:27:50.739037 29640 rpcz_store.cc:276] 0320 15:27:45.106302 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.106922 (+ 620us) service_pool.cc:225] Handling call
0320 15:27:50.738881 (+5631959us) ts_tablet_manager.cc:1151] Deleting tablet 3ed29146bff74e86a366d3c6329176f4
0320 15:27:50.738890 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.738944 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5631845}
W20260320 15:27:50.739763 28872 catalog_manager.cc:4969] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet 3ed29146bff74e86a366d3c6329176f4 because the tablet was not found. No further retry: Not found: Tablet not found: 3ed29146bff74e86a366d3c6329176f4
I20260320 15:27:50.740151 29640 tablet_replica.cc:333] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:50.740391 29640 raft_consensus.cc:2243] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.742558 30395 raft_consensus.cc:493] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.742715 30395 raft_consensus.cc:515] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.743607 30395 leader_election.cc:290] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.745136 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.745391 28440 raft_consensus.cc:2393] T 6bbc5d5f49a04216a352ddb1b571dce0 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80636bb608a1431b955576e296ff63bb in current term 1: Already voted for candidate 3ce5383d28b347ab979765b880e5773c in this term.
I20260320 15:27:50.734710 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.746276 28444 raft_consensus.cc:3060] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.746682 30955 tablet.cc:1723] T 325caab9b8f34f04a97b41137652ccdb P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.747226 30949 tablet.cc:1723] T 3f4a2c8ebc374de9bd8078a757c6be57 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.755681 30949 tablet.cc:1723] T 57cbefe3a19044f78b64db951df5f7c1 P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.757347 31006 tablet.cc:1723] T 3f4a2c8ebc374de9bd8078a757c6be57 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.758755 28425 tablet_service.cc:1511] Processing CreateTablet for tablet c1239b37b3fc482ebcf7e17037211591 (DEFAULT_TABLE table=default.Table_e96bc88284294560a8ef0fc23a1b34aa [id=708276ea8c974d3aa31e5abd81bdd923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.776396 28550 tablet_service.cc:1511] Processing CreateTablet for tablet c1239b37b3fc482ebcf7e17037211591 (DEFAULT_TABLE table=default.Table_e96bc88284294560a8ef0fc23a1b34aa [id=708276ea8c974d3aa31e5abd81bdd923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.777078 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1239b37b3fc482ebcf7e17037211591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.777674 28409 tablet_service.cc:1511] Processing CreateTablet for tablet fbfa09813bc7446eabe5c04718fb7731 (DEFAULT_TABLE table=default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.778429 28556 tablet_service.cc:1511] Processing CreateTablet for tablet fbfa09813bc7446eabe5c04718fb7731 (DEFAULT_TABLE table=default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.778884 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfa09813bc7446eabe5c04718fb7731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.784144 29643 ts_tablet_manager.cc:1929] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:50.786129 28560 tablet_service.cc:1511] Processing CreateTablet for tablet d6f07e96754b4c9bb593c1c487162ddf (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.786813 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f07e96754b4c9bb593c1c487162ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.787513 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:50.784621 28417 tablet_service.cc:1511] Processing CreateTablet for tablet d6f07e96754b4c9bb593c1c487162ddf (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.792083 29643 log.cc:1199] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/d5095b42b69d40539118c1f63a95f795
I20260320 15:27:50.788499 28444 raft_consensus.cc:2468] T f3f2e17c67c544e38840c5dd8dec32e5 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:50.795764 30655 consensus_queue.cc:1048] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.796784 28524 leader_election.cc:304] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.787932 30951 tablet.cc:1723] T 57cbefe3a19044f78b64db951df5f7c1 P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.797057 30655 raft_consensus.cc:2804] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.797194 30655 raft_consensus.cc:697] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Becoming Leader. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Running, Role: LEADER
I20260320 15:27:50.797434 30655 consensus_queue.cc:237] T f3f2e17c67c544e38840c5dd8dec32e5 P 80636bb608a1431b955576e296ff63bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.798115 30629 raft_consensus.cc:493] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce5383d28b347ab979765b880e5773c)
I20260320 15:27:50.798231 30629 raft_consensus.cc:515] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.798327 29643 ts_tablet_manager.cc:1950] T d5095b42b69d40539118c1f63a95f795 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:50.798743 30629 leader_election.cc:290] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.799089 28734 tablet_bootstrap.cc:492] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.800179 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfa09813bc7446eabe5c04718fb7731. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.800179 28524 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 8428019c3b074b17aec493c14cc2b911
I20260320 15:27:50.800585 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1239b37b3fc482ebcf7e17037211591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.800865 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f07e96754b4c9bb593c1c487162ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.801339 28734 tablet_bootstrap.cc:654] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.808229 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92ea5e924a734c8ba6dfb641359593a3" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.811874 28736 tablet_bootstrap.cc:492] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:50.812032 28736 ts_tablet_manager.cc:1403] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.202s user 0.003s sys 0.002s
I20260320 15:27:50.812739 28736 raft_consensus.cc:359] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.812918 28736 raft_consensus.cc:385] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.812968 28736 raft_consensus.cc:740] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.813153 28736 consensus_queue.cc:260] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.813642 28736 ts_tablet_manager.cc:1434] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:50.813985 28736 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:50.815402 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:50.815738 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:50.815784 30395 raft_consensus.cc:493] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.815936 30395 raft_consensus.cc:515] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.816591 30395 leader_election.cc:290] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.817322 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.822799 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 442) took 1538 ms. Trace:
I20260320 15:27:50.830122 29640 raft_consensus.cc:2272] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.831647 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:50.832013 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ab9893da469449bba556d11bd347d89b (table default.Table_f57610d7b6234bb9bcc7aa518b2bca0f [id=910ef5cd98e1495f829465985895c17b]). Expected version 1 got 0
I20260320 15:27:50.832724 28580 raft_consensus.cc:3060] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.838498 29640 ts_tablet_manager.cc:1916] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.845779 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:50.846135 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:50.849721 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.849944 28301 raft_consensus.cc:2468] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:50.850469 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2e17c67c544e38840c5dd8dec32e5" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:50.850677 28301 raft_consensus.cc:3060] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.853894 28301 raft_consensus.cc:2468] T f3f2e17c67c544e38840c5dd8dec32e5 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 1.
I20260320 15:27:50.854610 30773 raft_consensus.cc:493] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.854837 30773 raft_consensus.cc:515] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.855707 30773 leader_election.cc:290] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:50.846679 28383 leader_election.cc:343] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.857921 28383 leader_election.cc:304] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:50.859376 30964 raft_consensus.cc:2749] T 92ea5e924a734c8ba6dfb641359593a3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:50.866766 31028 tablet.cc:1723] T 008811c90aeb4ca4a4ad5c0190a60dcc P 80636bb608a1431b955576e296ff63bb: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.854362 31006 tablet.cc:1723] T 008811c90aeb4ca4a4ad5c0190a60dcc P d22e73471b9046d28d3cf63c905b11b7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:50.856329 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.867728 28446 raft_consensus.cc:2468] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:50.868265 28240 leader_election.cc:304] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.869007 30971 raft_consensus.cc:2804] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:50.869107 30971 raft_consensus.cc:493] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:50.869187 30971 raft_consensus.cc:3060] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.873220 29643 rpcz_store.cc:276] 0320 15:27:49.284057 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.284435 (+ 378us) service_pool.cc:225] Handling call
0320 15:27:50.366034 (+1081599us) ts_tablet_manager.cc:1151] Deleting tablet d5095b42b69d40539118c1f63a95f795
0320 15:27:50.366048 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.630441 (+264393us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.783971 (+153530us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:50.822767 (+ 38796us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1081396}
I20260320 15:27:50.873880 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet d5095b42b69d40539118c1f63a95f795 (table default.Table_8aca34fa920c499eb7e2ae000ced9d02 [id=e652fa47007e4ee3bf10ffb11ecc7f96]) successfully deleted
I20260320 15:27:50.876799 30971 raft_consensus.cc:515] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.877417 30971 leader_election.cc:290] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:50.878156 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:50.878331 28446 raft_consensus.cc:3060] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:50.883368 28446 raft_consensus.cc:2468] T fca60a0f372f4d1194d754d124b78ff8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:50.873769 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 444) took 1468 ms. Trace:
I20260320 15:27:50.884330 29643 rpcz_store.cc:276] 0320 15:27:49.405561 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.405622 (+ 61us) service_pool.cc:225] Handling call
0320 15:27:50.873622 (+1468000us) ts_tablet_manager.cc:1151] Deleting tablet 9d7f7b99899c4001adcbdce58910e717
0320 15:27:50.873637 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.873750 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1467787}
I20260320 15:27:50.884464 28240 leader_election.cc:304] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:50.885385 30852 raft_consensus.cc:2804] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:50.885536 30852 raft_consensus.cc:697] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
W20260320 15:27:50.885869 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 9d7f7b99899c4001adcbdce58910e717 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9d7f7b99899c4001adcbdce58910e717 already in progress: creating tablet
I20260320 15:27:50.885816 30852 consensus_queue.cc:237] T fca60a0f372f4d1194d754d124b78ff8 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.887346 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 2ce588403fc24b55a5a820e153a7336a (DEFAULT_TABLE table=default.Table_c535c1e04d8d45bc986fa72bbcf8babf [id=dd8f18887be842f894fdfb2242fd5b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.887800 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ce588403fc24b55a5a820e153a7336a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.888526 29643 tablet_replica.cc:333] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:50.888736 29643 raft_consensus.cc:2243] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:50.892514 28733 tablet_bootstrap.cc:492] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.892668 28733 ts_tablet_manager.cc:1403] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.226s user 0.041s sys 0.034s
I20260320 15:27:50.892767 28556 tablet_service.cc:1511] Processing CreateTablet for tablet b58cc1c7766c48cc8829f399afa7146f (DEFAULT_TABLE table=default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.893193 28556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58cc1c7766c48cc8829f399afa7146f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.893148 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 2ce588403fc24b55a5a820e153a7336a (DEFAULT_TABLE table=default.Table_c535c1e04d8d45bc986fa72bbcf8babf [id=dd8f18887be842f894fdfb2242fd5b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.893421 28733 raft_consensus.cc:359] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.893633 28733 raft_consensus.cc:385] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.893707 28733 raft_consensus.cc:740] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.894012 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ce588403fc24b55a5a820e153a7336a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.888890 29643 raft_consensus.cc:2272] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:50.890503 28409 tablet_service.cc:1511] Processing CreateTablet for tablet 612e44ad8b5746ebbade46293da7ca97 (DEFAULT_TABLE table=default.Table_1eeb969c59d948078955219bd038ce5b [id=e103280f1af24a0fb562bb08167e288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.898114 28409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e44ad8b5746ebbade46293da7ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.898353 29643 ts_tablet_manager.cc:1916] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:50.891335 28416 tablet_service.cc:1511] Processing CreateTablet for tablet b58cc1c7766c48cc8829f399afa7146f (DEFAULT_TABLE table=default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.900894 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58cc1c7766c48cc8829f399afa7146f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.894047 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 612e44ad8b5746ebbade46293da7ca97 (DEFAULT_TABLE table=default.Table_1eeb969c59d948078955219bd038ce5b [id=e103280f1af24a0fb562bb08167e288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.895162 28733 consensus_queue.cc:260] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.905630 28733 ts_tablet_manager.cc:1434] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260320 15:27:50.905943 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:50.906479 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (table default.Table_7ac4015d018a4c8e96ea949d122786a3 [id=7be232d8d79846eb858e6dd529427db4]). Expected version 1 got 0
I20260320 15:27:50.906571 28733 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.911309 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:50.911691 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:50.912002 28889 catalog_manager.cc:5671] T 8eb3d5e19cda48fda124b6238902c03a P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:50.912535 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:50.912794 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:50.913620 28889 catalog_manager.cc:5671] T eb011d672f244585b90272443897ed5a P 3ce5383d28b347ab979765b880e5773c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce5383d28b347ab979765b880e5773c (127.27.74.1). New cstate: current_term: 1 leader_uuid: "3ce5383d28b347ab979765b880e5773c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: HEALTHY } } }
I20260320 15:27:50.914099 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e44ad8b5746ebbade46293da7ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.914642 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:50.914969 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:50.916112 28733 tablet_bootstrap.cc:654] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.916415 28734 tablet_bootstrap.cc:492] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.916563 28734 ts_tablet_manager.cc:1403] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.118s user 0.005s sys 0.000s
I20260320 15:27:50.917843 28734 raft_consensus.cc:359] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.918082 28734 raft_consensus.cc:385] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.918148 28734 raft_consensus.cc:740] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.918437 28734 consensus_queue.cc:260] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:50.932395 28383 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.27.74.1:39627, user_credentials={real_user=slave}} blocked reactor thread for 73239.3us
I20260320 15:27:50.933600 28734 ts_tablet_manager.cc:1434] T 52ab0f33ad6b4cdfbe36e707872bee95 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.017s user 0.003s sys 0.000s
I20260320 15:27:50.936851 28733 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.937011 28733 ts_tablet_manager.cc:1403] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.007s
I20260320 15:27:50.937703 28733 raft_consensus.cc:359] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.937865 28733 raft_consensus.cc:385] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.937906 28733 raft_consensus.cc:740] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.938146 28733 consensus_queue.cc:260] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.939121 30866 raft_consensus.cc:493] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.939252 30866 raft_consensus.cc:515] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.939883 30866 leader_election.cc:290] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.940646 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:50.940824 28734 tablet_bootstrap.cc:492] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.941572 28733 ts_tablet_manager.cc:1434] T 4d12bd6ae67f4600a0d54e7d5d502dda P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260320 15:27:50.942862 28733 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.946806 28733 tablet_bootstrap.cc:654] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.951949 28734 tablet_bootstrap.cc:654] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.951861 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bbc5d5f49a04216a352ddb1b571dce0" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.965692 28736 tablet_bootstrap.cc:654] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.965626 28302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.966017 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:50.966485 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:50.966867 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:50.967048 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:50.971378 30952 tablet.cc:1723] T 3f4a2c8ebc374de9bd8078a757c6be57 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:50.974344 28734 tablet_bootstrap.cc:492] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:50.974517 28734 ts_tablet_manager.cc:1403] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.001s
I20260320 15:27:50.975270 28734 raft_consensus.cc:359] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.975510 28734 raft_consensus.cc:385] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.975617 28734 raft_consensus.cc:740] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.975857 28734 consensus_queue.cc:260] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.977876 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a61cfbf6eac4665a0b12ea7ccb9bb68" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:50.982635 28734 ts_tablet_manager.cc:1434] T 9732504551294b3bb3369cdaacacb8d3 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260320 15:27:50.982928 30773 consensus_queue.cc:1048] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:50.984113 28733 tablet_bootstrap.cc:492] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:50.984293 28733 ts_tablet_manager.cc:1403] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.011s
I20260320 15:27:50.984236 28268 tablet_service.cc:1511] Processing CreateTablet for tablet ec195522725942348e84236ac907be44 (DEFAULT_TABLE table=default.Table_4afe8b5be24444de80274a2c43ad204b [id=4a1edae7107844d5973fb94b141bb122]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.984737 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec195522725942348e84236ac907be44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.984802 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 1d1139da116240ff8b086f888905f953 (DEFAULT_TABLE table=default.Table_fa5403c78a5f4823959f74237f673da6 [id=951f820a52c44a3fad1675f2cc1be417]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.985108 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d1139da116240ff8b086f888905f953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.985100 28733 raft_consensus.cc:359] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.985271 28733 raft_consensus.cc:385] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:50.985325 28733 raft_consensus.cc:740] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:50.985564 28733 consensus_queue.cc:260] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:50.986416 28734 tablet_bootstrap.cc:492] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:50.986830 28733 ts_tablet_manager.cc:1434] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:50.987114 28733 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:50.987823 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:50.989954 28733 tablet_bootstrap.cc:654] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:50.991070 30964 raft_consensus.cc:493] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.991221 30964 raft_consensus.cc:515] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.991943 30964 leader_election.cc:290] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.992692 28268 tablet_service.cc:1511] Processing CreateTablet for tablet c1239b37b3fc482ebcf7e17037211591 (DEFAULT_TABLE table=default.Table_e96bc88284294560a8ef0fc23a1b34aa [id=708276ea8c974d3aa31e5abd81bdd923]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.992695 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94b4a6d75804c3e947af4f588ef8f5a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:50.993189 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1239b37b3fc482ebcf7e17037211591. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:50.993371 28382 leader_election.cc:343] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:50.994606 28281 tablet_service.cc:1511] Processing CreateTablet for tablet d6f07e96754b4c9bb593c1c487162ddf (DEFAULT_TABLE table=default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.995014 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6f07e96754b4c9bb593c1c487162ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.995178 30655 raft_consensus.cc:493] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:50.995321 30655 raft_consensus.cc:515] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:50.992744 28277 tablet_service.cc:1511] Processing CreateTablet for tablet fbfa09813bc7446eabe5c04718fb7731 (DEFAULT_TABLE table=default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:50.995800 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfa09813bc7446eabe5c04718fb7731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:50.995939 30655 leader_election.cc:290] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:50.999719 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:51.003306 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 2ce588403fc24b55a5a820e153a7336a (DEFAULT_TABLE table=default.Table_c535c1e04d8d45bc986fa72bbcf8babf [id=dd8f18887be842f894fdfb2242fd5b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.003865 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ce588403fc24b55a5a820e153a7336a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.004997 28733 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.005149 28733 ts_tablet_manager.cc:1403] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260320 15:27:51.005530 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:51.005785 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:51.005908 28733 raft_consensus.cc:359] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.006085 28733 raft_consensus.cc:385] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.006151 28733 raft_consensus.cc:740] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.006390 28733 consensus_queue.cc:260] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.007273 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 0acad11980f842e7b3a09216241e493d (table default.Table_fdf1117756d04248ad30f6a0c9101c6f [id=e703fc2e6e2345138d5a6852097ade20]). Expected version 1 got 0
I20260320 15:27:51.007753 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:51.007692 28276 tablet_service.cc:1511] Processing CreateTablet for tablet b58cc1c7766c48cc8829f399afa7146f (DEFAULT_TABLE table=default.Table_0f97e4f818004a0785b29e361c14a453 [id=4f9785bbfdf44be59e2362cd776d44e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.008062 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:51.008124 28276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b58cc1c7766c48cc8829f399afa7146f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.008978 28733 ts_tablet_manager.cc:1434] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260320 15:27:51.009462 28272 tablet_service.cc:1511] Processing CreateTablet for tablet 612e44ad8b5746ebbade46293da7ca97 (DEFAULT_TABLE table=default.Table_1eeb969c59d948078955219bd038ce5b [id=e103280f1af24a0fb562bb08167e288a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.009800 28733 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.009840 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612e44ad8b5746ebbade46293da7ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.011456 28733 tablet_bootstrap.cc:654] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.016387 28734 tablet_bootstrap.cc:654] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.018237 30964 raft_consensus.cc:493] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.018402 30725 consensus_queue.cc:1048] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.018412 30964 raft_consensus.cc:515] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.019054 30964 leader_election.cc:290] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.019737 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.019935 28580 raft_consensus.cc:2468] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:51.020334 28382 leader_election.cc:304] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.020630 30964 raft_consensus.cc:2804] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:51.020712 30964 raft_consensus.cc:493] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.020788 30964 raft_consensus.cc:3060] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:51.021241 28524 leader_election.cc:343] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:51.021461 28524 leader_election.cc:304] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.021844 30725 raft_consensus.cc:2749] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:51.023518 30964 raft_consensus.cc:515] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.024616 30964 leader_election.cc:290] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.024706 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:51.024889 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.025084 28574 raft_consensus.cc:2468] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:51.024896 28580 raft_consensus.cc:3060] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.028756 28736 tablet_bootstrap.cc:492] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.028770 28580 raft_consensus.cc:2468] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:51.028932 28736 ts_tablet_manager.cc:1403] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.215s user 0.003s sys 0.001s
I20260320 15:27:51.029309 28382 leader_election.cc:304] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.029693 28736 raft_consensus.cc:359] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.029870 28736 raft_consensus.cc:385] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.029925 28736 raft_consensus.cc:740] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.030151 28736 consensus_queue.cc:260] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.030855 28736 ts_tablet_manager.cc:1434] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:51.031224 28736 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.032006 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
W20260320 15:27:51.032373 28524 consensus_peers.cc:597] T d5095b42b69d40539118c1f63a95f795 P 80636bb608a1431b955576e296ff63bb -> Peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Couldn't send request to peer d22e73471b9046d28d3cf63c905b11b7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d5095b42b69d40539118c1f63a95f795. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:51.032428 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:51.033164 30964 raft_consensus.cc:2804] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:51.033387 30964 raft_consensus.cc:697] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:51.033373 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:51.033656 30964 consensus_queue.cc:237] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.033872 28889 catalog_manager.cc:5763] TS 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:51.034011 28736 tablet_bootstrap.cc:654] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.037307 29640 ts_tablet_manager.cc:1929] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:51.037487 29640 log.cc:1199] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/fcff253b9cce45b4ad11dc38397afee2
I20260320 15:27:51.037891 29640 ts_tablet_manager.cc:1950] T fcff253b9cce45b4ad11dc38397afee2 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.038300 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 328) took 5697 ms. Trace:
I20260320 15:27:51.038412 29640 rpcz_store.cc:276] 0320 15:27:45.340955 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.341015 (+ 60us) service_pool.cc:225] Handling call
0320 15:27:50.740076 (+5399061us) ts_tablet_manager.cc:1151] Deleting tablet fcff253b9cce45b4ad11dc38397afee2
0320 15:27:50.740093 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:50.842562 (+102469us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.037136 (+194574us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.038276 (+ 1140us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":89306,"tablet-delete.queue_time_us":5392081}
I20260320 15:27:51.039096 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet fcff253b9cce45b4ad11dc38397afee2 (table default.Table_591ab13c4a4a4aa9ba82b371492cab9e [id=b5695d7a1e4a4f8fb847ab0442e14c18]) successfully deleted
I20260320 15:27:51.040735 28734 tablet_bootstrap.cc:492] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.040894 28734 ts_tablet_manager.cc:1403] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.000s
I20260320 15:27:51.041651 28734 raft_consensus.cc:359] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.041841 28734 raft_consensus.cc:385] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.041899 28734 raft_consensus.cc:740] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.042138 28734 consensus_queue.cc:260] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.045089 28580 raft_consensus.cc:3060] T eb011d672f244585b90272443897ed5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.048046 28734 ts_tablet_manager.cc:1434] T 0a58d5350b8f407188b32786bfae2892 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260320 15:27:51.048862 30866 raft_consensus.cc:493] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.049011 30866 raft_consensus.cc:515] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.049515 28734 tablet_bootstrap.cc:492] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:51.049674 30866 leader_election.cc:290] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:51.051478 28734 tablet_bootstrap.cc:654] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
W20260320 15:27:51.051453 28524 leader_election.cc:343] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: db16d86664794305bde714a7ff4d5caf
I20260320 15:27:51.052136 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fca60a0f372f4d1194d754d124b78ff8" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:51.052345 28574 raft_consensus.cc:3060] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.055229 28574 raft_consensus.cc:2468] T fca60a0f372f4d1194d754d124b78ff8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:51.059873 28524 leader_election.cc:304] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.060472 30725 raft_consensus.cc:3060] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.063382 30725 raft_consensus.cc:2749] T 6bbc5d5f49a04216a352ddb1b571dce0 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260320 15:27:51.065120 28524 leader_election.cc:343] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: 8428019c3b074b17aec493c14cc2b911
I20260320 15:27:51.065295 28524 leader_election.cc:304] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.065900 30725 raft_consensus.cc:2749] T 8428019c3b074b17aec493c14cc2b911 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:51.068794 29643 ts_tablet_manager.cc:1929] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:51.069038 29643 log.cc:1199] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/1ccfd2d5e82448b99f1022954b98c6cc
I20260320 15:27:51.069516 29643 ts_tablet_manager.cc:1950] T 1ccfd2d5e82448b99f1022954b98c6cc P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:51.070480 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 445) took 1657 ms. Trace:
W20260320 15:27:51.071367 28524 leader_election.cc:343] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:51.073189 30725 consensus_queue.cc:1048] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.074069 28634 heartbeater.cc:499] Master 127.27.74.62:43951 was elected leader, sending a full tablet report...
I20260320 15:27:51.074828 28524 leader_election.cc:304] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.075393 30725 raft_consensus.cc:2749] T 3a61cfbf6eac4665a0b12ea7ccb9bb68 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260320 15:27:51.076838 29640 tablet_replica.cc:333] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.077064 29640 raft_consensus.cc:2243] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.077287 29640 raft_consensus.cc:2272] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.077939 29640 ts_tablet_manager.cc:1916] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.084228 28736 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.084419 28736 ts_tablet_manager.cc:1403] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.002s
I20260320 15:27:51.085345 28736 raft_consensus.cc:359] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.085534 28736 raft_consensus.cc:385] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.085592 28736 raft_consensus.cc:740] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.085848 28736 consensus_queue.cc:260] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.088209 28736 ts_tablet_manager.cc:1434] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260320 15:27:51.090221 29643 rpcz_store.cc:276] 0320 15:27:49.412674 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.412736 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:50.888462 (+1475726us) ts_tablet_manager.cc:1151] Deleting tablet 1ccfd2d5e82448b99f1022954b98c6cc
0320 15:27:50.888472 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.047695 (+159223us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.068655 (+ 20960us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.070458 (+ 1803us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1475522}
I20260320 15:27:51.091593 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 1ccfd2d5e82448b99f1022954b98c6cc (table default.Table_3588ac586adb42938b994103229f8dfb [id=8c7716828378437792822ef6d844a589]) successfully deleted
I20260320 15:27:51.092655 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 450) took 1241 ms. Trace:
I20260320 15:27:51.092833 29643 rpcz_store.cc:276] 0320 15:27:49.850868 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.850932 (+ 64us) service_pool.cc:225] Handling call
0320 15:27:51.092484 (+1241552us) ts_tablet_manager.cc:1151] Deleting tablet c37fc78b5d1546f69723cac88ce26ea8
0320 15:27:51.092503 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.092627 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1241293}
W20260320 15:27:51.093480 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet c37fc78b5d1546f69723cac88ce26ea8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c37fc78b5d1546f69723cac88ce26ea8 already in progress: creating tablet
I20260320 15:27:51.093969 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 451) took 1228 ms. Trace:
I20260320 15:27:51.094151 29643 rpcz_store.cc:276] 0320 15:27:49.865062 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.865116 (+ 54us) service_pool.cc:225] Handling call
0320 15:27:51.093842 (+1228726us) ts_tablet_manager.cc:1151] Deleting tablet 60ad9b381aaa47f1afb63df266faee09
0320 15:27:51.093852 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.093949 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1228508}
W20260320 15:27:51.094726 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 60ad9b381aaa47f1afb63df266faee09 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60ad9b381aaa47f1afb63df266faee09 already in progress: creating tablet
I20260320 15:27:51.095201 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 452) took 1214 ms. Trace:
I20260320 15:27:51.095331 29643 rpcz_store.cc:276] 0320 15:27:49.880494 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.880565 (+ 71us) service_pool.cc:225] Handling call
0320 15:27:51.095082 (+1214517us) ts_tablet_manager.cc:1151] Deleting tablet a1965a7f25ca4c4d95575a6c93ec376f
0320 15:27:51.095093 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.095183 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1214285}
W20260320 15:27:51.095878 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet a1965a7f25ca4c4d95575a6c93ec376f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1965a7f25ca4c4d95575a6c93ec376f already in progress: creating tablet
I20260320 15:27:51.096323 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 453) took 1201 ms. Trace:
I20260320 15:27:51.097281 29643 rpcz_store.cc:276] 0320 15:27:49.894437 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.894494 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:51.096186 (+1201692us) ts_tablet_manager.cc:1151] Deleting tablet bf37dbb8bc8e48c5a11f106ecd93dd78
0320 15:27:51.096198 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.096304 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1201480}
I20260320 15:27:51.097502 28574 raft_consensus.cc:3060] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:51.097873 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet bf37dbb8bc8e48c5a11f106ecd93dd78 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf37dbb8bc8e48c5a11f106ecd93dd78 already in progress: creating tablet
I20260320 15:27:51.098301 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 454) took 1116 ms. Trace:
I20260320 15:27:51.098421 29643 rpcz_store.cc:276] 0320 15:27:49.982027 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:49.982084 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:51.098180 (+1116096us) ts_tablet_manager.cc:1151] Deleting tablet 55bd68c6e4364a038a815d1fc05ad25e
0320 15:27:51.098192 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.098283 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1115918}
W20260320 15:27:51.098923 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 55bd68c6e4364a038a815d1fc05ad25e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55bd68c6e4364a038a815d1fc05ad25e already in progress: creating tablet
I20260320 15:27:51.099277 29643 tablet_replica.cc:333] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:51.099551 29643 raft_consensus.cc:2243] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:51.107918 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d94b4a6d75804c3e947af4f588ef8f5a" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:51.108633 28736 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.110543 28736 tablet_bootstrap.cc:654] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.118847 29640 ts_tablet_manager.cc:1929] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:51.119027 29640 log.cc:1199] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/cc97a5f2bd964a05bb7b2427226debdd
I20260320 15:27:51.119292 30852 raft_consensus.cc:493] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.119436 30852 raft_consensus.cc:515] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.120769 30852 leader_election.cc:290] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.124387 28733 tablet_bootstrap.cc:492] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.124601 28733 ts_tablet_manager.cc:1403] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.115s user 0.017s sys 0.022s
I20260320 15:27:51.125351 28733 raft_consensus.cc:359] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.125581 28733 raft_consensus.cc:385] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.125674 28733 raft_consensus.cc:740] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.125998 28733 consensus_queue.cc:260] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.127272 28733 ts_tablet_manager.cc:1434] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:51.127946 28733 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.101075 31029 raft_consensus.cc:493] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.128216 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:51.128283 31029 raft_consensus.cc:515] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.128948 31029 leader_election.cc:290] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.120699 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:51.129145 28440 raft_consensus.cc:2468] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:51.129580 28240 leader_election.cc:304] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.130041 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6498952e19c45dbacb1fa7af8ece8f8" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:51.130424 28580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f94140a5e8b4a06890bca86456d64a6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.130764 30852 raft_consensus.cc:2804] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260320 15:27:51.130905 28382 leader_election.cc:343] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:51.130971 30852 raft_consensus.cc:493] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.131047 30852 raft_consensus.cc:3060] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.132786 28733 tablet_bootstrap.cc:654] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.134277 30852 raft_consensus.cc:515] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.134974 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f94140a5e8b4a06890bca86456d64a6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:51.135202 28297 raft_consensus.cc:2468] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 0.
I20260320 15:27:51.135411 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:51.135644 28440 raft_consensus.cc:3060] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.136188 28297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb3d5e19cda48fda124b6238902c03a" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:51.138461 28440 raft_consensus.cc:2468] T 3ba3def9ab7641d2ad4a6d0538bf978a P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:51.138916 28240 leader_election.cc:304] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.139146 30971 raft_consensus.cc:2804] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:51.139266 30971 raft_consensus.cc:697] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:51.139519 30971 consensus_queue.cc:237] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.141343 30952 tablet.cc:1723] T 57cbefe3a19044f78b64db951df5f7c1 P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260320 15:27:51.143780 30852 leader_election.cc:290] T 3ba3def9ab7641d2ad4a6d0538bf978a P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.145455 29643 raft_consensus.cc:2272] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.148408 29643 ts_tablet_manager.cc:1916] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.148886 28733 tablet_bootstrap.cc:492] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.148815 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 9a43e22e642841e0943bbb0d9b4c9389 (DEFAULT_TABLE table=default.Table_ac17b8c1958d407e938050586e69cb01 [id=cad0f5251fdc43e5930579bbe96bf9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.149065 28733 ts_tablet_manager.cc:1403] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.006s
I20260320 15:27:51.149222 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a43e22e642841e0943bbb0d9b4c9389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.150030 28733 raft_consensus.cc:359] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.150239 28733 raft_consensus.cc:385] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.150341 28733 raft_consensus.cc:740] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.150627 28733 consensus_queue.cc:260] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.151584 28733 ts_tablet_manager.cc:1434] T 50c1398534d84701ae7085a037f236ad P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260320 15:27:51.152019 28733 tablet_bootstrap.cc:492] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.154523 28733 tablet_bootstrap.cc:654] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.156440 28734 tablet_bootstrap.cc:492] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.156612 28734 ts_tablet_manager.cc:1403] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.107s user 0.004s sys 0.000s
I20260320 15:27:51.157339 28734 raft_consensus.cc:359] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.157516 28734 raft_consensus.cc:385] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.157583 28734 raft_consensus.cc:740] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.157817 28734 consensus_queue.cc:260] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.158615 28734 ts_tablet_manager.cc:1434] T 7840b529206d482283cc550d2c5e7606 P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:51.158898 28734 tablet_bootstrap.cc:492] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:51.160933 28734 tablet_bootstrap.cc:654] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.161194 28736 tablet_bootstrap.cc:492] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.161386 28736 ts_tablet_manager.cc:1403] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.001s
I20260320 15:27:51.162237 28736 raft_consensus.cc:359] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.162489 28736 raft_consensus.cc:385] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.162540 28736 raft_consensus.cc:740] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.163239 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 9a43e22e642841e0943bbb0d9b4c9389 (DEFAULT_TABLE table=default.Table_ac17b8c1958d407e938050586e69cb01 [id=cad0f5251fdc43e5930579bbe96bf9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.163811 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a43e22e642841e0943bbb0d9b4c9389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.164510 28416 tablet_service.cc:1511] Processing CreateTablet for tablet 67203fb2d81e4416b89d84586532c827 (DEFAULT_TABLE table=default.Table_b1165742e11d4e3cb02367b20289687f [id=e65abc18967d4baeb08a8f9794d55f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.164964 28416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67203fb2d81e4416b89d84586532c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.162757 28736 consensus_queue.cc:260] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.185079 28425 tablet_service.cc:1511] Processing CreateTablet for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0 (DEFAULT_TABLE table=default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.185590 28425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.187294 28417 tablet_service.cc:1511] Processing CreateTablet for tablet 2d8282bdae434fc9a28a69f5ca056b35 (DEFAULT_TABLE table=default.Table_757caf8aa06441c4838e2e0c6a585dbb [id=5fac67c34eb34108808f97e1e07850eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.187728 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d8282bdae434fc9a28a69f5ca056b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.187981 28550 tablet_service.cc:1511] Processing CreateTablet for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0 (DEFAULT_TABLE table=default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.188632 28550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.192384 28560 tablet_service.cc:1511] Processing CreateTablet for tablet 2d8282bdae434fc9a28a69f5ca056b35 (DEFAULT_TABLE table=default.Table_757caf8aa06441c4838e2e0c6a585dbb [id=5fac67c34eb34108808f97e1e07850eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.192848 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d8282bdae434fc9a28a69f5ca056b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.208746 30971 consensus_queue.cc:1048] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.210291 28733 tablet_bootstrap.cc:492] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.210660 28733 ts_tablet_manager.cc:1403] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.059s user 0.007s sys 0.022s
I20260320 15:27:51.211410 28733 raft_consensus.cc:359] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.211660 28733 raft_consensus.cc:385] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.211724 28733 raft_consensus.cc:740] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.211998 28733 consensus_queue.cc:260] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.212538 28733 ts_tablet_manager.cc:1434] T a185f84c24ea402684aa1cc3628d4f11 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:51.212872 28733 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.214745 28733 tablet_bootstrap.cc:654] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.215893 28734 tablet_bootstrap.cc:492] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.215405 28417 tablet_service.cc:1511] Processing CreateTablet for tablet f74a78aaa7644cb2930cd0b53d911561 (DEFAULT_TABLE table=default.Table_0ec651f9c0604b82a569f618b6f06b79 [id=8503710392dc46469ad097a2ed922ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.216058 28734 ts_tablet_manager.cc:1403] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.000s
I20260320 15:27:51.216327 28417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74a78aaa7644cb2930cd0b53d911561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.216755 28734 raft_consensus.cc:359] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.216943 28734 raft_consensus.cc:385] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.216996 28734 raft_consensus.cc:740] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.217231 28734 consensus_queue.cc:260] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.218318 28734 ts_tablet_manager.cc:1434] T c217b54730ea46c88ec01a03755b83cc P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:51.219025 28734 tablet_bootstrap.cc:492] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:51.228158 28736 ts_tablet_manager.cc:1434] T a6498952e19c45dbacb1fa7af8ece8f8 P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.067s user 0.001s sys 0.001s
I20260320 15:27:51.228547 30725 raft_consensus.cc:493] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.228746 30725 raft_consensus.cc:515] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.229400 30725 leader_election.cc:290] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:51.229537 28733 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.229688 28733 ts_tablet_manager.cc:1403] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260320 15:27:51.230443 28733 raft_consensus.cc:359] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.230643 28733 raft_consensus.cc:385] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.230767 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e779a274913f4caeaf82e3f180457830" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:51.230851 28733 raft_consensus.cc:740] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.230993 28440 raft_consensus.cc:2468] T e779a274913f4caeaf82e3f180457830 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:51.231099 28733 consensus_queue.cc:260] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.231395 28736 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.231758 28733 ts_tablet_manager.cc:1434] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:51.231951 28524 leader_election.cc:304] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80636bb608a1431b955576e296ff63bb, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.232077 28733 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.232633 30725 raft_consensus.cc:2804] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:51.232789 30725 raft_consensus.cc:493] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.232969 30725 raft_consensus.cc:3060] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.233923 28562 tablet_service.cc:1511] Processing CreateTablet for tablet 67203fb2d81e4416b89d84586532c827 (DEFAULT_TABLE table=default.Table_b1165742e11d4e3cb02367b20289687f [id=e65abc18967d4baeb08a8f9794d55f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.234086 28733 tablet_bootstrap.cc:654] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.239877 28562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67203fb2d81e4416b89d84586532c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.237170 28736 tablet_bootstrap.cc:654] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.242125 28296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e779a274913f4caeaf82e3f180457830" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c" is_pre_election: true
I20260320 15:27:51.247457 28296 raft_consensus.cc:2468] T e779a274913f4caeaf82e3f180457830 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80636bb608a1431b955576e296ff63bb in term 0.
I20260320 15:27:51.235867 29640 ts_tablet_manager.cc:1950] T cc97a5f2bd964a05bb7b2427226debdd P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.250929 28734 tablet_bootstrap.cc:654] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.251824 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 329) took 5876 ms. Trace:
I20260320 15:27:51.251941 29640 rpcz_store.cc:276] 0320 15:27:45.375613 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.375664 (+ 51us) service_pool.cc:225] Handling call
0320 15:27:51.076728 (+5701064us) ts_tablet_manager.cc:1151] Deleting tablet cc97a5f2bd964a05bb7b2427226debdd
0320 15:27:51.076738 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.115072 (+ 38334us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.118670 (+ 3598us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.251798 (+133128us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5699993}
W20260320 15:27:51.253373 28383 leader_election.cc:343] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:51.253520 28383 leader_election.cc:304] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22e73471b9046d28d3cf63c905b11b7; no voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb
I20260320 15:27:51.253903 31029 raft_consensus.cc:2749] T d94b4a6d75804c3e947af4f588ef8f5a P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:51.254083 29640 tablet_replica.cc:333] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.254313 29640 raft_consensus.cc:2243] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.254495 29640 raft_consensus.cc:2272] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:51.254833 28383 consensus_peers.cc:597] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 -> Peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Couldn't send request to peer 3ce5383d28b347ab979765b880e5773c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e3f6d9a528946908891f77f951d23dd. This is attempt 1: this message will repeat every 5th retry.
I20260320 15:27:51.255110 29640 ts_tablet_manager.cc:1916] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.256367 28560 tablet_service.cc:1511] Processing CreateTablet for tablet f74a78aaa7644cb2930cd0b53d911561 (DEFAULT_TABLE table=default.Table_0ec651f9c0604b82a569f618b6f06b79 [id=8503710392dc46469ad097a2ed922ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.256384 31029 consensus_queue.cc:1048] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.256928 28560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74a78aaa7644cb2930cd0b53d911561. 1 dirs total, 0 dirs full, 0 dirs failed
W20260320 15:27:51.257162 28383 leader_election.cc:343] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
W20260320 15:27:51.257791 28383 leader_election.cc:343] T a6498952e19c45dbacb1fa7af8ece8f8 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:51.259713 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.260123 28574 raft_consensus.cc:2468] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:51.252451 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet cc97a5f2bd964a05bb7b2427226debdd (table default.Table_55bb102a048d48b59fe62e863b719d76 [id=921ff2ab0e8d4f148ff576400523ec06]) successfully deleted
I20260320 15:27:51.264503 28383 leader_election.cc:304] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:51.264634 28524 leader_election.cc:304] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.265100 31029 raft_consensus.cc:2804] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:51.265106 30655 raft_consensus.cc:2749] T a6498952e19c45dbacb1fa7af8ece8f8 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:51.265180 31029 raft_consensus.cc:493] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.265236 31029 raft_consensus.cc:3060] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.206406 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 325caab9b8f34f04a97b41137652ccdb (table default.Table_929744d40f1a48c5aa6e0d16d97e93c3 [id=4685dae9caa842d6bc5fe1c294a37af3]). Expected version 1 got 0
I20260320 15:27:51.265837 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3c7a605bdcf14d2ab64ad078efd3d999 (table default.Table_1590be7e3ed84ee2b141222c66efbe1c [id=210d7611e226430b8db648a92580ccc6]). Expected version 1 got 0
I20260320 15:27:51.266110 29643 ts_tablet_manager.cc:1929] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:51.266214 29643 log.cc:1199] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/9dd293a68f894cbfa122d53918816c28
I20260320 15:27:51.266564 29643 ts_tablet_manager.cc:1950] T 9dd293a68f894cbfa122d53918816c28 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:51.267589 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 455) took 1246 ms. Trace:
I20260320 15:27:51.267683 29643 rpcz_store.cc:276] 0320 15:27:50.021081 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:50.021160 (+ 79us) service_pool.cc:225] Handling call
0320 15:27:51.099225 (+1078065us) ts_tablet_manager.cc:1151] Deleting tablet 9dd293a68f894cbfa122d53918816c28
0320 15:27:51.099234 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.234717 (+135483us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.266000 (+ 31283us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.267496 (+ 1496us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1077811}
I20260320 15:27:51.268172 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 9dd293a68f894cbfa122d53918816c28 (table default.Table_726c3589aba84426980572e969866fa3 [id=0466f754fc5f43249331fda298bb726c]) successfully deleted
I20260320 15:27:51.268555 31029 raft_consensus.cc:515] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.269467 29643 tablet_replica.cc:333] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:51.269695 28906 catalog_manager.cc:5671] T 4bd1799035a3431092fbdfad3c42d22b P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:51.271194 28736 tablet_bootstrap.cc:492] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.271144 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f94140a5e8b4a06890bca86456d64a6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:51.271462 28736 ts_tablet_manager.cc:1403] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260320 15:27:51.272147 28733 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.272285 28733 ts_tablet_manager.cc:1403] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.003s
I20260320 15:27:51.272653 28736 raft_consensus.cc:359] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.272889 28736 raft_consensus.cc:385] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.272993 28736 raft_consensus.cc:740] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.272954 28524 leader_election.cc:304] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.273020 28733 raft_consensus.cc:359] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.273180 28733 raft_consensus.cc:385] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.273237 28733 raft_consensus.cc:740] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.273351 30655 raft_consensus.cc:2749] T 8eb3d5e19cda48fda124b6238902c03a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:51.273464 28736 consensus_queue.cc:260] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.273464 28733 consensus_queue.cc:260] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.273991 28736 ts_tablet_manager.cc:1434] T d04ab44039124f36ac60601098586b4e P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260320 15:27:51.274395 28733 ts_tablet_manager.cc:1434] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260320 15:27:51.274932 28736 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.275126 28733 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.276752 28733 tablet_bootstrap.cc:654] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.276690 28574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba3def9ab7641d2ad4a6d0538bf978a" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb"
I20260320 15:27:51.277868 28574 raft_consensus.cc:3060] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.279778 28736 tablet_bootstrap.cc:654] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.269853 31029 leader_election.cc:290] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:51.271699 28382 leader_election.cc:343] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): Illegal state: must be running to vote when last-logged opid is not known
I20260320 15:27:51.278646 28734 tablet_bootstrap.cc:492] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.282583 28734 ts_tablet_manager.cc:1403] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.000s
I20260320 15:27:51.282771 28574 raft_consensus.cc:2468] T 3ba3def9ab7641d2ad4a6d0538bf978a P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:51.283885 29643 raft_consensus.cc:2243] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:51.284313 29643 raft_consensus.cc:2272] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:51.284288 28524 leader_election.cc:343] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627): Not found: Tablet not found: db16d86664794305bde714a7ff4d5caf
I20260320 15:27:51.284410 28524 leader_election.cc:304] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80636bb608a1431b955576e296ff63bb; no voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.285025 30629 raft_consensus.cc:2749] T db16d86664794305bde714a7ff4d5caf P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260320 15:27:51.285267 29643 ts_tablet_manager.cc:1916] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.286828 28734 raft_consensus.cc:359] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.287012 28734 raft_consensus.cc:385] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.287066 28734 raft_consensus.cc:740] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.287321 28734 consensus_queue.cc:260] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.287894 29640 ts_tablet_manager.cc:1929] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:51.288107 28734 ts_tablet_manager.cc:1434] T c456e9eff7904004830bcda3efa77dbf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260320 15:27:51.288532 28734 tablet_bootstrap.cc:492] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:51.289395 28733 tablet_bootstrap.cc:492] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.289783 28733 ts_tablet_manager.cc:1403] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260320 15:27:51.290514 28734 tablet_bootstrap.cc:654] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.290689 28733 raft_consensus.cc:359] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.290956 28733 raft_consensus.cc:385] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.291061 28733 raft_consensus.cc:740] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.291366 28733 consensus_queue.cc:260] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.292728 28733 ts_tablet_manager.cc:1434] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260320 15:27:51.293071 28733 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.293955 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 9a43e22e642841e0943bbb0d9b4c9389 (DEFAULT_TABLE table=default.Table_ac17b8c1958d407e938050586e69cb01 [id=cad0f5251fdc43e5930579bbe96bf9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.294091 29640 log.cc:1199] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/4190329df1b644c392ff8cbace9aae7a
I20260320 15:27:51.294389 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a43e22e642841e0943bbb0d9b4c9389. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.295866 29643 ts_tablet_manager.cc:1929] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:51.295972 29643 log.cc:1199] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/0e3f6d9a528946908891f77f951d23dd
I20260320 15:27:51.296325 29643 ts_tablet_manager.cc:1950] T 0e3f6d9a528946908891f77f951d23dd P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:51.297097 29643 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55030 (request call id 456) took 1196 ms. Trace:
I20260320 15:27:51.297178 29643 rpcz_store.cc:276] 0320 15:27:50.100409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:50.100471 (+ 62us) service_pool.cc:225] Handling call
0320 15:27:51.269413 (+1168942us) ts_tablet_manager.cc:1151] Deleting tablet 0e3f6d9a528946908891f77f951d23dd
0320 15:27:51.269425 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.291351 (+ 21926us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.295749 (+ 4398us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.297076 (+ 1327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1168735}
I20260320 15:27:51.297940 29643 tablet_replica.cc:333] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: stopping tablet replica
I20260320 15:27:51.298123 29643 raft_consensus.cc:2243] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.298286 29643 raft_consensus.cc:2272] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.298830 29643 ts_tablet_manager.cc:1916] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.299753 30852 raft_consensus.cc:493] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.299898 30852 raft_consensus.cc:515] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.300522 30852 leader_election.cc:290] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
W20260320 15:27:51.303364 28240 leader_election.cc:343] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b16ffb28029e4b7ca37ce36efe9e0271
I20260320 15:27:51.297679 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet 0e3f6d9a528946908891f77f951d23dd (table default.Table_50cb6ab7e16b477dbb7fec9d108e6d48 [id=40645833d1084e6d94ae0169a116576b]) successfully deleted
I20260320 15:27:51.303992 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4bd1799035a3431092fbdfad3c42d22b (table default.Table_a89976557ff942d7a9b2412273ac664b [id=63cb5bcef9f34041beff3f359eb8b923]). Expected version 1 got 0
I20260320 15:27:51.304713 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f6240d441c624b238f7c13eb3c5fa9fc (table default.Table_43739f94773040b8a2e5986104bf32fc [id=cf9c764d6b874bd1861111099488895e]). Expected version 1 got 0
I20260320 15:27:51.305019 28906 catalog_manager.cc:5671] T 813604f6c6c747eda18f7a01d9d8ec97 P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:51.305402 28268 tablet_service.cc:1511] Processing CreateTablet for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0 (DEFAULT_TABLE table=default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.305563 28281 tablet_service.cc:1511] Processing CreateTablet for tablet 67203fb2d81e4416b89d84586532c827 (DEFAULT_TABLE table=default.Table_b1165742e11d4e3cb02367b20289687f [id=e65abc18967d4baeb08a8f9794d55f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.305828 28268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3285b579325e4ed0ba2f97a1a2e2c4d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.305896 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 52ab0f33ad6b4cdfbe36e707872bee95 (table default.Table_20b641db350b494dbd1c458752f5a7b9 [id=e6144852663f4116bf276bf082e4c4e7]). Expected version 2 got 0
I20260320 15:27:51.306100 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 71db28879e554f578b8a7b0e14419ab7 (table default.Table_303ce9b5065e40fcaf9800c531b31dee [id=348a5cfdd28b45f0b2ea54926047ddc7]). Expected version 1 got 0
I20260320 15:27:51.306166 28281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67203fb2d81e4416b89d84586532c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.306241 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ab9893da469449bba556d11bd347d89b (table default.Table_f57610d7b6234bb9bcc7aa518b2bca0f [id=910ef5cd98e1495f829465985895c17b]). Expected version 1 got 0
I20260320 15:27:51.306370 28277 tablet_service.cc:1511] Processing CreateTablet for tablet 2d8282bdae434fc9a28a69f5ca056b35 (DEFAULT_TABLE table=default.Table_757caf8aa06441c4838e2e0c6a585dbb [id=5fac67c34eb34108808f97e1e07850eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.306437 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6725124b62ad4171a2cfe072687b8c3d (table default.Table_da8719eda33f44a4a0bd0f5fe838515f [id=65f2ef6dd1884f6e974f3b1c77e1b567]). Expected version 1 got 0
I20260320 15:27:51.306670 28277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d8282bdae434fc9a28a69f5ca056b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.306943 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet d94b4a6d75804c3e947af4f588ef8f5a (table default.Table_592f5890923d4f61aad0b52ea7be4aeb [id=f0bd5f1c14fb4bc9be3b8d94ddcaa953]). Expected version 3 got 0
I20260320 15:27:51.307142 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 6bbc5d5f49a04216a352ddb1b571dce0 (table default.Table_94ef14573dca40e19391aaab0e148383 [id=b67cf3894d2a412d9e2a5e9d4eb2280d]). Expected version 1 got 0
I20260320 15:27:51.307358 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet a8ac11b0ca134c948fdd9ff4b114e6aa (table default.Table_7ac4015d018a4c8e96ea949d122786a3 [id=7be232d8d79846eb858e6dd529427db4]). Expected version 1 got 0
I20260320 15:27:51.307914 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet f8f70142c3a449439a0a475bddafdd40 (table default.Table_4df4defc37874a3aa612d777259d9d11 [id=01294509a81747008737b6c3ffae50b4]). Expected version 2 got 0
I20260320 15:27:51.308207 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 806d635bb5594f0bb826b9b31468b485 (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]). Expected version 2 got 0
I20260320 15:27:51.308475 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 4b527973ede040e286f7b8f2e7536699 (table default.Table_aff011445f764f9abb08fc4c8d68fc4d [id=7621ebb93211458ba6572228ebfddee4]). Expected version 1 got 0
I20260320 15:27:51.308600 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 8eb3d5e19cda48fda124b6238902c03a (table default.Table_58424e8752fe4921bafb617ee7eb20de [id=72049ded61454044b534b8470b2f9db5]). Expected version 1 got 0
I20260320 15:27:51.308741 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet fca60a0f372f4d1194d754d124b78ff8 (table default.Table_afb7c7729a7c4fff902d08c295e69c39 [id=fd1279c22df9494eb690d1cbcef3f340]). Expected version 2 got 0
I20260320 15:27:51.309473 28733 tablet_bootstrap.cc:654] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.309548 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0f1b495cd3cd4fc1a9d6a2fc12d5bb2e (table default.Table_fb53bcadbad44ef0a0c0e7169d29bbfd [id=035b44fe37d94257babdcd3ffe38d068]). Expected version 1 got 0
I20260320 15:27:51.309662 29640 ts_tablet_manager.cc:1950] T 4190329df1b644c392ff8cbace9aae7a P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.310490 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 330) took 5575 ms. Trace:
I20260320 15:27:51.310599 29640 rpcz_store.cc:276] 0320 15:27:45.734702 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.734953 (+ 251us) service_pool.cc:225] Handling call
0320 15:27:51.254016 (+5519063us) ts_tablet_manager.cc:1151] Deleting tablet 4190329df1b644c392ff8cbace9aae7a
0320 15:27:51.254029 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.280260 (+ 26231us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.287778 (+ 7518us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.310467 (+ 22689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5518824}
I20260320 15:27:51.310950 29640 tablet_replica.cc:333] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.311124 29640 raft_consensus.cc:2243] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.311256 29640 raft_consensus.cc:2272] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.311753 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 4190329df1b644c392ff8cbace9aae7a (table default.Table_b2fb7e79b74940ce87fe158e71ae31db [id=39d38f795d004300a287cdb264a1f8ca]) successfully deleted
I20260320 15:27:51.312103 29640 ts_tablet_manager.cc:1916] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.319283 29640 ts_tablet_manager.cc:1929] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:51.319414 29640 log.cc:1199] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/d04ab44039124f36ac60601098586b4e
I20260320 15:27:51.324261 29643 ts_tablet_manager.cc:1929] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260320 15:27:51.324558 29643 log.cc:1199] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-1/wal/wals/f59a3c9fa17a4babaa9a18f18d51f193
I20260320 15:27:51.325099 29643 ts_tablet_manager.cc:1950] T f59a3c9fa17a4babaa9a18f18d51f193 P d22e73471b9046d28d3cf63c905b11b7: Deleting consensus metadata
I20260320 15:27:51.326395 28873 catalog_manager.cc:5002] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): tablet f59a3c9fa17a4babaa9a18f18d51f193 (table default.Table_0e217a9219f04e1cb94b0c63324773a1 [id=4737d6ecec7041afb0192ab248a093ca]) successfully deleted
I20260320 15:27:51.327490 28272 tablet_service.cc:1511] Processing CreateTablet for tablet f74a78aaa7644cb2930cd0b53d911561 (DEFAULT_TABLE table=default.Table_0ec651f9c0604b82a569f618b6f06b79 [id=8503710392dc46469ad097a2ed922ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260320 15:27:51.328013 28272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f74a78aaa7644cb2930cd0b53d911561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260320 15:27:51.328096 29640 ts_tablet_manager.cc:1950] T d04ab44039124f36ac60601098586b4e P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.328907 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 331) took 5569 ms. Trace:
I20260320 15:27:51.329000 29640 rpcz_store.cc:276] 0320 15:27:45.759964 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:45.760021 (+ 57us) service_pool.cc:225] Handling call
0320 15:27:51.310902 (+5550881us) ts_tablet_manager.cc:1151] Deleting tablet d04ab44039124f36ac60601098586b4e
0320 15:27:51.310911 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.316158 (+ 5247us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.319173 (+ 3015us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.328886 (+ 9713us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5550628}
I20260320 15:27:51.329340 29640 tablet_replica.cc:333] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.329530 29640 raft_consensus.cc:2243] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 1 LEADER]: Raft consensus shutting down.
I20260320 15:27:51.329782 28736 tablet_bootstrap.cc:492] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.329818 29640 raft_consensus.cc:2272] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.329916 28736 ts_tablet_manager.cc:1403] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.055s user 0.004s sys 0.000s
I20260320 15:27:51.330336 29640 ts_tablet_manager.cc:1916] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.330612 28736 raft_consensus.cc:359] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.330771 28736 raft_consensus.cc:385] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.330819 28736 raft_consensus.cc:740] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.330927 31020 consensus_queue.cc:1048] T 26ae379304674bc3941af125f0d9d6b3 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.330992 28736 consensus_queue.cc:260] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.331516 28736 ts_tablet_manager.cc:1434] T 4d12bd6ae67f4600a0d54e7d5d502dda P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:51.332469 28736 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.333674 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet d04ab44039124f36ac60601098586b4e (table default.Table_c75cd37c416143ef883cf261b71cff1d [id=ab492077e90e4c0b8fde3b376bcc2568]) successfully deleted
I20260320 15:27:51.335846 28736 tablet_bootstrap.cc:654] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.340307 28734 tablet_bootstrap.cc:492] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.340463 28734 ts_tablet_manager.cc:1403] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.002s
I20260320 15:27:51.341228 28734 raft_consensus.cc:359] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.341411 28734 raft_consensus.cc:385] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.341457 28734 raft_consensus.cc:740] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.341706 28734 consensus_queue.cc:260] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } }
I20260320 15:27:51.342486 28734 ts_tablet_manager.cc:1434] T 7e9705754d6c4ab8a2f674dc359df83f P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:51.342926 28734 tablet_bootstrap.cc:492] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
I20260320 15:27:51.345080 28734 tablet_bootstrap.cc:654] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.349823 28733 tablet_bootstrap.cc:492] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.350004 28733 ts_tablet_manager.cc:1403] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.057s user 0.001s sys 0.013s
I20260320 15:27:51.350761 28733 raft_consensus.cc:359] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.350952 28733 raft_consensus.cc:385] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.351008 28733 raft_consensus.cc:740] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.351233 28733 consensus_queue.cc:260] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.352320 28733 ts_tablet_manager.cc:1434] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260320 15:27:51.352756 28733 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.354965 28733 tablet_bootstrap.cc:654] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.357601 30971 consensus_queue.cc:1048] T 6bbc5d5f49a04216a352ddb1b571dce0 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.359642 28582 raft_consensus.cc:3060] T afb692aab24849bc8e50086927421e13 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.359973 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 31379c9e2adc4fd48e0d0b6e9443615a (table default.Table_08b880856b7f455ca758812764146420 [id=55547a21133840a3a80a02db2c72676d]). Expected version 1 got 0
I20260320 15:27:51.360237 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 3f4a2c8ebc374de9bd8078a757c6be57 (table default.Table_5d8b2e640491465da035984a69b16fcc [id=ca21c08f2e724a7b8e136248fcecd103]). Expected version 2 got 0
I20260320 15:27:51.360538 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 7bb680bb0863485aaad85a749221e81b (table default.Table_a9b80338873049fbaaa6c166ee20099e [id=0998e8ce9720447caf0d5e7fbf145fe7]). Expected version 1 got 0
I20260320 15:27:51.360682 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 366df9b29cee4b2b9dfd72b0061f175a (table default.Table_c78295faeae84addb0a4a445fef9bfbd [id=73b84a8ea2d14d62ab611990602bcec5]). Expected version 1 got 0
I20260320 15:27:51.361148 29640 ts_tablet_manager.cc:1929] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260320 15:27:51.361253 29640 log.cc:1199] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/90cd18e2ccbf48c8b804329bd3abf847
I20260320 15:27:51.361548 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 514e7b02a28142c2b371644089044a9d (table default.Table_910f9050e6d24365b23da47d9569910a [id=0d1bce42ec29458a95121f253235d22f]). Expected version 1 got 0
I20260320 15:27:51.362218 28906 catalog_manager.cc:5671] T 76379bf0fa554820be020f647582a72b P 80636bb608a1431b955576e296ff63bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80636bb608a1431b955576e296ff63bb (127.27.74.3). New cstate: current_term: 1 leader_uuid: "80636bb608a1431b955576e296ff63bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } health_report { overall_health: UNKNOWN } } }
I20260320 15:27:51.362900 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 76379bf0fa554820be020f647582a72b (table default.Table_72b122270f4542a19e5b00a85e754f16 [id=d84ac38833fe43cfbaefb2f47f7a3af4]). Expected version 1 got 0
I20260320 15:27:51.363170 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 26ae379304674bc3941af125f0d9d6b3 (table default.Table_223074fa3ea142debceae9aae8ce1ce1 [id=9941875dffce49d680808963e123cdbd]). Expected version 1 got 0
I20260320 15:27:51.363330 28575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b16ffb28029e4b7ca37ce36efe9e0271" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.363869 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 321fe367ec3348f8a049167f3f8c15d5 (table default.Table_96e808cfcc8442e2bce7aa1f13986b23 [id=c5d8e4f7585a4d72af6d2af8cff2950e]). Expected version 1 got 0
I20260320 15:27:51.364112 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 0acad11980f842e7b3a09216241e493d (table default.Table_fdf1117756d04248ad30f6a0c9101c6f [id=e703fc2e6e2345138d5a6852097ade20]). Expected version 1 got 0
I20260320 15:27:51.364548 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 008811c90aeb4ca4a4ad5c0190a60dcc (table default.Table_952147a3f089453295332fa44b0bf633 [id=f7db6a0e41de4c1da8c3ce8fea5fb4ad]). Expected version 1 got 0
I20260320 15:27:51.364919 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet ef0d80754dfd4bbd9817f568063511ef (table default.Table_9a40cefec6fa4bb9b70743b12241badd [id=a242e635105f44caa8322297dff1f7f2]). Expected version 1 got 0
I20260320 15:27:51.364908 29640 ts_tablet_manager.cc:1950] T 90cd18e2ccbf48c8b804329bd3abf847 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.363507 28575 raft_consensus.cc:2468] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:51.366959 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 339) took 5168 ms. Trace:
I20260320 15:27:51.367053 29640 rpcz_store.cc:276] 0320 15:27:46.198420 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.212790 (+ 14370us) service_pool.cc:225] Handling call
0320 15:27:51.329292 (+5116502us) ts_tablet_manager.cc:1151] Deleting tablet 90cd18e2ccbf48c8b804329bd3abf847
0320 15:27:51.329301 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.346370 (+ 17069us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.361031 (+ 14661us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.366938 (+ 5907us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5115358}
I20260320 15:27:51.367496 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet 90cd18e2ccbf48c8b804329bd3abf847 (table default.Table_ed5104d8fb4f435e87290e89a34f96e1 [id=2b46d17aa5134f4da389e33d36671984]) successfully deleted
I20260320 15:27:51.367786 30725 raft_consensus.cc:515] T e779a274913f4caeaf82e3f180457830 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.368503 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 338) took 5186 ms. Trace:
I20260320 15:27:51.368605 29640 rpcz_store.cc:276] 0320 15:27:46.198246 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.200111 (+ 1865us) service_pool.cc:225] Handling call
0320 15:27:51.368378 (+5168267us) ts_tablet_manager.cc:1151] Deleting tablet d823ab41194542fba5540e53e9f4680b
0320 15:27:51.368390 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.368486 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5154222}
W20260320 15:27:51.369061 28872 catalog_manager.cc:4982] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): delete failed for tablet d823ab41194542fba5540e53e9f4680b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d823ab41194542fba5540e53e9f4680b already in progress: creating tablet
I20260320 15:27:51.369356 29640 tablet_replica.cc:333] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.369591 29640 raft_consensus.cc:2243] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.369735 29640 raft_consensus.cc:2272] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Raft consensus is shut down!
I20260320 15:27:51.370070 31020 raft_consensus.cc:493] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.370209 31020 raft_consensus.cc:515] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.370343 28301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f94140a5e8b4a06890bca86456d64a6" candidate_uuid: "d22e73471b9046d28d3cf63c905b11b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce5383d28b347ab979765b880e5773c"
I20260320 15:27:51.370463 28301 raft_consensus.cc:3060] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.370821 31020 leader_election.cc:290] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.370889 29640 ts_tablet_manager.cc:1916] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.371939 28440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef0d80754dfd4bbd9817f568063511ef" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:51.372133 28440 raft_consensus.cc:2468] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 0.
I20260320 15:27:51.372582 28240 leader_election.cc:304] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
I20260320 15:27:51.372834 30852 raft_consensus.cc:2804] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:51.372893 30852 raft_consensus.cc:493] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.372946 30852 raft_consensus.cc:3060] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.373339 28301 raft_consensus.cc:2468] T 7f94140a5e8b4a06890bca86456d64a6 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22e73471b9046d28d3cf63c905b11b7 in term 1.
I20260320 15:27:51.375118 30395 raft_consensus.cc:493] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260320 15:27:51.375272 30395 raft_consensus.cc:515] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.375720 30952 tablet.cc:1723] T 008811c90aeb4ca4a4ad5c0190a60dcc P 3ce5383d28b347ab979765b880e5773c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260320 15:27:51.375881 30395 leader_election.cc:290] T 7f94140a5e8b4a06890bca86456d64a6 P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:51.377104 28736 tablet_bootstrap.cc:492] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c: No bootstrap required, opened a new log
I20260320 15:27:51.377245 28736 ts_tablet_manager.cc:1403] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.000s
I20260320 15:27:51.377315 28733 tablet_bootstrap.cc:492] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.377431 28733 ts_tablet_manager.cc:1403] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260320 15:27:51.378057 28733 raft_consensus.cc:359] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.378075 28736 raft_consensus.cc:359] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.378221 28733 raft_consensus.cc:385] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.378257 28736 raft_consensus.cc:385] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.378276 28733 raft_consensus.cc:740] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.378362 28736 raft_consensus.cc:740] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.378527 28733 consensus_queue.cc:260] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.378564 28736 consensus_queue.cc:260] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.379101 28736 ts_tablet_manager.cc:1434] T 09c87a35c8fd4d87a1cd9e5ca3db15ec P 3ce5383d28b347ab979765b880e5773c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260320 15:27:51.379145 28733 ts_tablet_manager.cc:1434] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260320 15:27:51.379395 28733 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.379429 28736 tablet_bootstrap.cc:492] T a185f84c24ea402684aa1cc3628d4f11 P 3ce5383d28b347ab979765b880e5773c: Bootstrap starting.
I20260320 15:27:51.379997 28582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef0d80754dfd4bbd9817f568063511ef" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80636bb608a1431b955576e296ff63bb" is_pre_election: true
I20260320 15:27:51.380631 28383 leader_election.cc:304] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters: 80636bb608a1431b955576e296ff63bb
I20260320 15:27:51.381244 28736 tablet_bootstrap.cc:654] T a185f84c24ea402684aa1cc3628d4f11 P 3ce5383d28b347ab979765b880e5773c: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.381410 31029 raft_consensus.cc:2804] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:51.381505 28733 tablet_bootstrap.cc:654] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.381544 31029 raft_consensus.cc:697] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [term 1 LEADER]: Becoming Leader. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Running, Role: LEADER
I20260320 15:27:51.381841 31029 consensus_queue.cc:237] T 7f94140a5e8b4a06890bca86456d64a6 P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.382896 30395 raft_consensus.cc:493] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22e73471b9046d28d3cf63c905b11b7)
I20260320 15:27:51.383028 28734 tablet_bootstrap.cc:492] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7: No bootstrap required, opened a new log
I20260320 15:27:51.383028 30395 raft_consensus.cc:515] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.383150 28734 ts_tablet_manager.cc:1403] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7: Time spent bootstrapping tablet: real 0.040s user 0.003s sys 0.000s
I20260320 15:27:51.383633 30395 leader_election.cc:290] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 3ce5383d28b347ab979765b880e5773c (127.27.74.1:39627)
I20260320 15:27:51.383625 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f94140a5e8b4a06890bca86456d64a6" candidate_uuid: "80636bb608a1431b955576e296ff63bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7" is_pre_election: true
I20260320 15:27:51.383949 28734 raft_consensus.cc:359] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.384099 28734 raft_consensus.cc:385] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.384151 28734 raft_consensus.cc:740] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22e73471b9046d28d3cf63c905b11b7, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.384330 28734 consensus_queue.cc:260] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.385941 28734 ts_tablet_manager.cc:1434] T 956494c03c36462da65323aeccf65ccf P d22e73471b9046d28d3cf63c905b11b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260320 15:27:51.386225 28734 tablet_bootstrap.cc:492] T 11033b84b4234b6995c528c6d869cf5f P d22e73471b9046d28d3cf63c905b11b7: Bootstrap starting.
W20260320 15:27:51.391480 28524 leader_election.cc:343] T 4cfad5cf843a437e8e15c7236d1dc5fe P 80636bb608a1431b955576e296ff63bb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: 4cfad5cf843a437e8e15c7236d1dc5fe
I20260320 15:27:51.392778 31020 consensus_queue.cc:1048] T afb692aab24849bc8e50086927421e13 P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.394196 31020 consensus_queue.cc:1048] T 50c1398534d84701ae7085a037f236ad P 3ce5383d28b347ab979765b880e5773c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.394721 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 57cbefe3a19044f78b64db951df5f7c1 (table default.Table_a32711390c664fe387579cb227f11e85 [id=61789d27ebb643599e793253bcbdd4c4]). Expected version 2 got 0
I20260320 15:27:51.394066 28239 leader_election.cc:304] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, 80636bb608a1431b955576e296ff63bb; no voters: d22e73471b9046d28d3cf63c905b11b7
I20260320 15:27:51.395197 28733 tablet_bootstrap.cc:492] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.395339 28733 ts_tablet_manager.cc:1403] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260320 15:27:51.395509 31020 raft_consensus.cc:2804] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260320 15:27:51.395659 31020 raft_consensus.cc:493] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260320 15:27:51.395946 31020 raft_consensus.cc:3060] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.396200 28734 tablet_bootstrap.cc:654] T 11033b84b4234b6995c528c6d869cf5f P d22e73471b9046d28d3cf63c905b11b7: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.396283 28733 raft_consensus.cc:359] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.396473 28733 raft_consensus.cc:385] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260320 15:27:51.396586 28733 raft_consensus.cc:740] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80636bb608a1431b955576e296ff63bb, State: Initialized, Role: FOLLOWER
I20260320 15:27:51.396871 28906 catalog_manager.cc:5763] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463) does not have the latest schema for tablet 91f393d2ca304b899f788add43da7e9a (table default.Table_add1cf33998542f5afe4fd5216b91bf6 [id=258c68c3e8474fa3b0e198cbcf15c41e]). Expected version 1 got 0
I20260320 15:27:51.396860 28733 consensus_queue.cc:260] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.398957 28733 ts_tablet_manager.cc:1434] T ef0d80754dfd4bbd9817f568063511ef P 80636bb608a1431b955576e296ff63bb: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260320 15:27:51.399365 28904 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 6cc7dc78a25c4f32a511e2ed20a51aa3
I20260320 15:27:51.399989 28904 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:33394 (request call id 207) took 1342 ms. Trace:
I20260320 15:27:51.400262 28733 tablet_bootstrap.cc:492] T d94b4a6d75804c3e947af4f588ef8f5a P 80636bb608a1431b955576e296ff63bb: Bootstrap starting.
I20260320 15:27:51.400390 30852 raft_consensus.cc:515] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.400113 28904 rpcz_store.cc:276] 0320 15:27:50.057074 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:50.563046 (+505972us) service_pool.cc:225] Handling call
0320 15:27:50.563740 (+ 694us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0320 15:27:50.563939 (+ 199us) catalog_manager.cc:2815] Modifying in-memory table state
0320 15:27:50.564005 (+ 66us) catalog_manager.cc:2824] Locking tablets
0320 15:27:50.633387 (+ 69382us) catalog_manager.cc:2838] Removing table and tablets from system table
0320 15:27:50.646170 (+ 12783us) catalog_manager.cc:2856] Removing table from by-name map
0320 15:27:50.646281 (+ 111us) catalog_manager.cc:2872] Committing in-memory state
0320 15:27:51.399282 (+753001us) catalog_manager.cc:2876] Aborting table tasks
0320 15:27:51.399942 (+ 660us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0320 15:27:50.634736 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0320 15:27:50.634885 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0320 15:27:50.634893 (+ 8us) write_op.cc:435] Acquired schema lock
0320 15:27:50.634897 (+ 4us) tablet.cc:661] Decoding operations
0320 15:27:50.635030 (+ 133us) write_op.cc:620] Acquiring the partition lock for write op
0320 15:27:50.635070 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0320 15:27:50.635075 (+ 5us) tablet.cc:684] Acquiring locks for 2 operations
0320 15:27:50.635194 (+ 119us) tablet.cc:700] Row locks acquired
0320 15:27:50.635197 (+ 3us) write_op.cc:260] PREPARE: finished
0320 15:27:50.635396 (+ 199us) write_op.cc:270] Start()
0320 15:27:50.635447 (+ 51us) write_op.cc:276] Timestamp: P: 1774020470635196 usec, L: 0
0320 15:27:50.635452 (+ 5us) op_driver.cc:348] REPLICATION: starting
0320 15:27:50.635735 (+ 283us) log.cc:844] Serialized 725 byte log entry
0320 15:27:50.645124 (+ 9389us) op_driver.cc:464] REPLICATION: finished
0320 15:27:50.645266 (+ 142us) write_op.cc:301] APPLY: starting
0320 15:27:50.645286 (+ 20us) tablet.cc:1366] starting BulkCheckPresence
0320 15:27:50.645338 (+ 52us) tablet.cc:1369] finished BulkCheckPresence
0320 15:27:50.645365 (+ 27us) tablet.cc:1371] starting ApplyRowOperation cycle
0320 15:27:50.645447 (+ 82us) tablet.cc:1382] finished ApplyRowOperation cycle
0320 15:27:50.645485 (+ 38us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0320 15:27:50.645492 (+ 7us) write_op.cc:312] APPLY: finished
0320 15:27:50.645597 (+ 105us) log.cc:844] Serialized 29 byte log entry
0320 15:27:50.645792 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:50.645819 (+ 27us) write_op.cc:454] Released schema lock
0320 15:27:50.645827 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0320 15:27:50.645918 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0320 15:27:50.645922 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":92,"apply.run_cpu_time_us":637,"apply.run_wall_time_us":637,"num_ops":2,"prepare.queue_time_us":301,"prepare.run_cpu_time_us":1149,"prepare.run_wall_time_us":1151,"raft.queue_time_us":508,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":323,"replication_time_us":9636}]]}
I20260320 15:27:51.401139 30852 leader_election.cc:290] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.401558 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef0d80754dfd4bbd9817f568063511ef" candidate_uuid: "3ce5383d28b347ab979765b880e5773c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22e73471b9046d28d3cf63c905b11b7"
I20260320 15:27:51.401770 28444 raft_consensus.cc:3060] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 0 FOLLOWER]: Advancing to term 1
I20260320 15:27:51.402031 28733 tablet_bootstrap.cc:654] T d94b4a6d75804c3e947af4f588ef8f5a P 80636bb608a1431b955576e296ff63bb: Neither blocks nor log segments found. Creating new log.
I20260320 15:27:51.404250 28562 tablet_service.cc:1558] Processing DeleteTablet for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:50 UTC) from {username='slave'} at 127.0.0.1:36478
I20260320 15:27:51.404558 28444 raft_consensus.cc:2468] T ef0d80754dfd4bbd9817f568063511ef P d22e73471b9046d28d3cf63c905b11b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce5383d28b347ab979765b880e5773c in term 1.
I20260320 15:27:51.405131 28416 tablet_service.cc:1558] Processing DeleteTablet for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-03-20 15:27:50 UTC) from {username='slave'} at 127.0.0.1:55030
I20260320 15:27:51.405364 29640 ts_tablet_manager.cc:1929] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260320 15:27:51.405483 29640 log.cc:1199] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Deleting WAL directory at /tmp/dist-test-taskc3SESu/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1774020433812185-27944-0/minicluster-data/ts-2/wal/wals/b16ffb28029e4b7ca37ce36efe9e0271
I20260320 15:27:51.405788 29640 ts_tablet_manager.cc:1950] T b16ffb28029e4b7ca37ce36efe9e0271 P 80636bb608a1431b955576e296ff63bb: Deleting consensus metadata
I20260320 15:27:51.405807 28240 leader_election.cc:304] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce5383d28b347ab979765b880e5773c, d22e73471b9046d28d3cf63c905b11b7; no voters:
W20260320 15:27:51.406404 28873 catalog_manager.cc:4982] TS d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): delete failed for tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6cc7dc78a25c4f32a511e2ed20a51aa3 already in progress: creating tablet
I20260320 15:27:51.406580 29640 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36478 (request call id 340) took 5191 ms. Trace:
I20260320 15:27:51.406659 29640 rpcz_store.cc:276] 0320 15:27:46.214904 (+ 0us) service_pool.cc:168] Inserting onto call queue
0320 15:27:46.217991 (+ 3087us) service_pool.cc:225] Handling call
0320 15:27:51.369308 (+5151317us) ts_tablet_manager.cc:1151] Deleting tablet b16ffb28029e4b7ca37ce36efe9e0271
0320 15:27:51.369318 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0320 15:27:51.386422 (+ 17104us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.403904 (+ 17482us) tablet_metadata.cc:641] Metadata flushed
0320 15:27:51.406560 (+ 2656us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5128030}
I20260320 15:27:51.407088 30852 raft_consensus.cc:2804] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Leader election won for term 1
I20260320 15:27:51.407140 28872 catalog_manager.cc:5002] TS 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463): tablet b16ffb28029e4b7ca37ce36efe9e0271 (table default.Table_9b2e32517f9d49a9a117be78ffddb91e [id=5df94a1a6fc04a4983dc2888ef98c2d6]) successfully deleted
I20260320 15:27:51.407313 30852 raft_consensus.cc:697] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [term 1 LEADER]: Becoming Leader. State: Replica: 3ce5383d28b347ab979765b880e5773c, State: Running, Role: LEADER
I20260320 15:27:51.407706 30852 consensus_queue.cc:237] T ef0d80754dfd4bbd9817f568063511ef P 3ce5383d28b347ab979765b880e5773c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
I20260320 15:27:51.408972 28582 raft_consensus.cc:3060] T aba79ddde6e842dfb67597c5d9e371cb P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Advancing to term 1
W20260320 15:27:51.412451 28890 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad6d3aae8e75428c90ed2b3265316f57 consensus state has no leader
I20260320 15:27:51.413683 31029 consensus_queue.cc:1048] T aba79ddde6e842dfb67597c5d9e371cb P d22e73471b9046d28d3cf63c905b11b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260320 15:27:51.414830 31020 raft_consensus.cc:515] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7" member_type: VOTER last_known_addr { host: "127.27.74.2" port: 45233 } } peers { permanent_uuid: "3ce5383d28b347ab979765b880e5773c" member_type: VOTER last_known_addr { host: "127.27.74.1" port: 39627 } } peers { permanent_uuid: "80636bb608a1431b955576e296ff63bb" member_type: VOTER last_known_addr { host: "127.27.74.3" port: 44463 } }
W20260320 15:27:51.415588 28905 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 068515f077624aed83309109eb59d077 consensus state has no leader
I20260320 15:27:51.416007 29640 tablet_replica.cc:333] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: stopping tablet replica
I20260320 15:27:51.416275 29640 raft_consensus.cc:2243] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260320 15:27:51.416501 29640 raft_consensus.cc:2272] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260320 15:27:51.416683 28240 leader_election.cc:343] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233): Not found: Tablet not found: b16ffb28029e4b7ca37ce36efe9e0271
I20260320 15:27:51.417212 29640 ts_tablet_manager.cc:1916] T b4cad900411e4c9686701aa99206dd23 P 80636bb608a1431b955576e296ff63bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260320 15:27:51.417920 28900 catalog_manager.cc:5958] T 00000000000000000000000000000000 P ee1caebf640d4ec0bf9b28c4f496ca18: Sending DeleteTablet for 3 replicas of tablet 01828bedfdd94d62b780a9d5a40b17a5
I20260320 15:27:51.420048 28733 tablet_bootstrap.cc:492] T d94b4a6d75804c3e947af4f588ef8f5a P 80636bb608a1431b955576e296ff63bb: No bootstrap required, opened a new log
I20260320 15:27:51.420203 28733 ts_tablet_manager.cc:1403] T d94b4a6d75804c3e947af4f588ef8f5a P 80636bb608a1431b955576e296ff63bb: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20260320 15:27:51.420641 31020 leader_election.cc:290] T b16ffb28029e4b7ca37ce36efe9e0271 P 3ce5383d28b347ab979765b880e5773c [CANDIDATE]: Term 1 election: Requested vote from peers d22e73471b9046d28d3cf63c905b11b7 (127.27.74.2:45233), 80636bb608a1431b955576e296ff63bb (127.27.74.3:44463)
I20260320 15:27:51.421226 28733 raft_consensus.cc:359] T d94b4a6d75804c3e947af4f588ef8f5a P 80636bb608a1431b955576e296ff63bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22e73471b9046d28d3cf63c905b11b7"